Prepare merging ntfsrecover.h into logfile.h by adding a usn field to
RESTART_PAGE_HEADER. As this changes the record size, ignore the new
field in existing code.
Usually, only a few pages of the Windows log file are saved in an
ntfsclone image. This is inappropriate for building reference images
for recovering the log, and the --full-logfile option serves that
purpose.
When an INDX or MFT record could not be read while undoing the creation
of this record, there is nothing to do. However if this was undoing the
deletion of the last entry in an index, a new void index block has to be
created.
The previous fix for the warning referred to 'prevbuf' being used
uninitialized and this is also what the compiler says. However
initializing 'prevbuf' doesn't make the warning go away and further
testing revealed that it is really 'savebuf' being possibly used prior
to initialization that is the source of the warning (the incorrect
warning message is probably an optimization-related gcc bug). So replace
previous ineffective fix with explicit initialization of 'savebuf'.
For 64-bit (e.g. x86_64) Linux the 64-bit wide types resolve to long,
not long long as is the case in 32-bit (e.g. i386) Linux. So we need an
explicit cast to long long for 64-bit types since the format string must
specify the 'll' modifier in order to print 64-bit values.
Some compilers issue a warning when a pointer is initialized in
both alternatives of a condition. Force an extra initialization
to avoid such warnings.
Closing the volume is the way to sync the MFT to disk. When not doing
so, the MFT runlists in $DATA and $Bitmap are not synced if they have
been updated in the second resizing stage relative to runlists which
have grown outside their original MFT record.
Unlike in most cases, the bad sector inode has to be closed if it
was updated and required MFT extents (when there are a lot of bad
sectors and some of them were outside the truncated partition).
Not doing so causes the inode to not be fully synced to device.
The UTF-16LE label buffer containing the result of mbs2ucs is the one
that should be NULL-terminated when the label is longer than permitted.
Not the input buffer, which is a function parameter assumed to be
NULL-terminated anyway.
This is done to match the type of the LSN struct members in layout.h.
The effect of this change is that while these members were declared with
the le64 type previously, leLSN resolves to sle64. I.e. what was
previously unsigned fields are now signed.
Following this change we also need to switch over a few macros from
unsigned to signed versions in the code that uses these struct
definitions.
There were multiple cases of little-endian fields being used as
CPU-endian without byte swapping. This would result in incorrect
behaviour on big-endian systems.
On big-endian systems the result of the '!=' operation would be
endian-swapped rather than the first argument (which must have been the
intended action).
This is harmless except when we do strict endianness checking, in which
case this results in a compile error. Fixed by converting values to
CPU endianness before comparing them.
In 'dump_resident_attr_val', 'i' was sometimes used as a native-endian
'int'-precision string length value and sometimes used as a little-
endian 16-bit flags value. This type of mixed usage is bad practice and
results in a hard error when strict endianness checking is used.
Fixed by introducing new variable 'flags' to hold the little-endian 16-
bit flags value.
If the attribute type is specified by the user, 'attr_type' was assigned
a CPU-endian value, however if the attribute type was not specified it
would be assigned the attribute type AT_DATA, which is a little-endian
value. The rest of the code seems to assume that 'attr_type' is
CPU-endian, so this is clearly a bug.
Resolved by fixing the endianness of the variable at little-endian,
converting the input value to little-endian when specified.
In 'dump_attr_record' the variable 'u' was first used to store a
CPU-endian 32-bit value, and then to store a 16-bit little-endian value.
This is bad practice and results in a hard error when strict endian type
checking is used.
Fixed by storing the 16-bit little-endian flags value in a new variable
'flags'.
This commit addresses issues where little-endian variables are emitted
raw to a log or output stream which is to be interpreted by the user.
Outputting data in non-native endianness can cause confusion for anybody
attempting to debug issues with a file system.
If start buffer is more recent than restart, we update committed LSN
with last record LSN of block (last_end_lsn) while applying action but
forget about it while printing records with -f for investigation
purpose.
Note that while applying actions we use start_buffer to calculate
latest page out of block 2 and block 3 and then from latest take
committed LSN. For -f we don't need buffers so we just compare
directly with committed LSN from restart.
(contributed by Rakesh Pandit)
The new compression formats used by Windows 10 uses reparse data, and
a new reparse tag which it is useful to define even though these formats
is not yet supported by ntfs-3g.
This fixes the case where the original bad cluster list requires extents.
The list is processed globally, no relocation is done, and the list is
truncated, possibly fitting into fewer extents.
When block 2 or block 3 points backward to block 4, it is not clear
whether the log file only consists of block 2 or block 3 or the log
file has just wrapped around. The latter is now assumed.
When the bad cluster list required extent, ntfsclone and ntfsresize
did not process the extents, leading to unexpected read errors and
unmatching bitmaps. This fix enables the full list to be taken into
account.
The MFT has two runlists which may be partially stored in extents.
When these runlists have to be relocated, the relocations must be done
after the old runlists are not needed any more to read the data of
standard files, but before the MFT may be needed to extend the runlists
of standard files. Before doing so the MFT runlists have to be refreshed
from device in order to collect the updates which cannot be done in
memory during the first stage.
ntfsrecover applies to the metadata the updates which were requested on
Windows but could not be completed because they were interrupted by
some event such as a power failure, a hardware crash, a software crash
or the device being unplugged. Doing so, the file system is restored
to the latest consistent state.
No update to libntfs-3g is required by this implementation.
When used with the "no-action" option, the test for self-located MFT
still reported the partition to have been repaired. Adapt the report to
only tell repairing is possible.
In an NTFS boot sector, the first sector of the partition is limited to
32 bits and it may overflow on large disks. This field is only used for
booting on the partition and it is ignored by ntfs-3g, but the warning
in mkntfs mislead users, so improve it.
Unused entries in the MFT may have a bad length leading to fetch fixups
from unallocated memory. Check the condition, but do not wipe, leave it
to chkdsk to decide what should be fixed.
When the unmounting of the partition fails after running ntfsfix, a
second try was attempted. This cannot be done and leads to more errors
because some essential records have been freed, so better quit without
making a second try.
When the new option --unused-fast is used, clusters which appear as wiped
are not written again. This is useful for avoiding virtual partitions to
be extended to their full size.
Contributed by michael
ntfsfallocate is the equivalent of fallocate(1) :
Usage : ntfsfallocate [-n] [-p] [-o offset] -l length filename
For now, it is only released as a quarantined ntfsprogs because it can
produce configurations which cause subsequent updates of the file by
Windows to hang.
Distributions have complained about releasing a non-functional ntfsck.
Actually, ntfsck and a few other developer-oriented utilities were not
meant to be released by distributions and are only compiled in if the
configure option --enable-extra is set, and, for some reason, this
option is set by most distributions.
In order to get distributions to not complain, though making the source
code available for candidate developers to improve it, the non functional
or developer-oriented utilities (ntfsck, ntfsdump_logfile, ntfsmftalloc
and ntfsmove) are now only compiled in if the configure option
--enable-quarantined is set.
The ntfsprogs used to return failure when option --version or --help
was used, and this has triggered complains from distribution packagers
who use these options in packaging scripts.
With this patch, success is returned (same behavior as gcc).
When the partition is resized, the global bitmap size is adjusted accordingly,
however so far the new size was not set into the parent index (only minor
consequences).
The ioctl() function is not implemented in the Windows variant. By forcing
a negative apparent return, an alternative is triggered to get the
partition size which is normally obtained from an ioctl().
In some rare situations relocated runlists are longer than the original
ones and do not fit into the same extent. When this happens the runlist
updating is delayed and done globally. Be sure to use the updated global
bitmap for making the needed allocations.
X509 certificates have a purpose field restricting what the certificate
can be used for, and EFS encryption is such a purpose. Allow EFS encryption
to be at any position in the list.
When ntfsclone'ing to a file, the target file was truncated to the volume
size. This is not useful on file systems which support sparse files. In
the case of ntfs-3g this leads to prevent optimizations specific to
appending data. So when a sparse output file is detected, it is emptied
to benefit from subsequent appending of data.
The upcoming libgrypt-1.6 drops the "module" interface which was used
by ntfsdecrypt for decrypting files which were encrypted with the "DESX"
algorithm. This algorithm is a Microsoft variant of DES with a key size
of 128 bits, and is not natively supported by libgrypt. The module interface
made possible to declare an external algorithm so that all the encryption
modes could be processed the same way whether the algorithm was internal
or external.
This patch makes DESX a specific case, so that the module interface is
not needed any more. It is compatible with current libgrypt and upcoming
libgrypt-1.6
Jean-Pierre André
Erik Larsson