Cross platform coding: data type issues to remember.

1. long data type.
Always remember on Windows long is 4 bytes in size. On Linux it is 8 bytes!

2. Reading unsigned short variable.

void foo(char *shortString)
{
    unsigned short val;
    sscanf(shortString, "%u", &val);
}

This code works properly on Ubuntu. But gives core dump on CentOS / RHEL.
Never read a short variable using “%d” or “%u”, always use “%h” or “%hu”

    // this is the right thing to do.
    sscanf(shortString, "%hu", &val);
Advertisements

Use of FSCTL_GET_VOLUME_BITMAP to read $bitmap file from NTFS / ReFS volume

Q. How to read $bitmap file from NTFS / ReFS volume?

Ans:

FSCTL_GET_VOLUME_BITMAP IOCTL is provided to read the $bitmap file. To use this IOCT you should know the size of $bitmap in advance, If you do not know this then you’ll have to read $bitmap file in chunks and store it in file to refer it later. The sample pseudo code for reading $bitmap file looks like below:

Additional things to look for structures: STARTING_LCN_INPUT_BUFFER and VOLUME_BITMAP_BUFFER

 

#define BITMAP_CHUNK_SIZE 32*1024	// 32 KB is our chunk size to read $bitmap file

int
readVolumeBitmap(
    std::wstring volumeName
    )
{
	HANDLE hVolume;
    STARTING_LCN_INPUT_BUFFER startingLcn;
	VOLUME_BITMAP_BUFFER *volBitmap;
    UINT32 bitmapSize;
    DWORD bytesReturned, bytesWritten;
    BOOL ret, retFile;
    HANDLE hFile;
    std::wstring bitmapFile;
    int result;
	
	/* Open the volume for reading */
	hVolume = CreateFile(
                           volumeName.c_str(),
                           GENERIC_READ,
                           FILE_SHARE_READ | FILE_SHARE_WRITE,
                           NULL,
                           OPEN_EXISTING,
                           0,
                           NULL);

    startingLcn.StartingLcn.QuadPart = 0;
    bitmapSize = BITMAP_CHUNK_SIZE + sizeof(LARGE_INTEGER)*2;
    volBitmap = (VOLUME_BITMAP_BUFFER *) malloc (bitmapSize);

	bitmapFile = L"dummy_bitmap.bin";

    hFile = CreateFile(
                       bitmapFile.c_str(),
                       GENERIC_WRITE,
                       FILE_SHARE_READ | FILE_SHARE_WRITE,
                       NULL,
                       CREATE_ALWAYS,
                       0,
                       NULL);


	/* Read $bitmap file in loop */
    while (TRUE) {
        ret = DeviceIoControl (
                               hVolume,
                               FSCTL_GET_VOLUME_BITMAP,
                               &startingLcn,
                               sizeof(STARTING_LCN_INPUT_BUFFER),
                               volBitmap,
                               bitmapSize,
                               &bytesReturned,
                               NULL);

        if (startingLcn.StartingLcn.QuadPart != m_volBitmap->StartingLcn.QuadPart) {
                /* some ioctl error */
				return -1;
        }
        if (FALSE==ret){
			/* only error we expect is to read more data */
            if (GetLastError() != ERROR_MORE_DATA) {
                /* some ioctl error */
				return -1;
            }
        }

		/* find out the exact bytes read from $bitmap */
        bytesReturned -= sizeof(LARGE_INTEGER)*2;
        retFile = WriteFile (
                             hFile,
                             volBitmap->Buffer,
                             bytesReturned,
                             &bytesWritten,
                             NULL);
		/* If eof occured for reading $bitmap break the loop */
        if (TRUE==ret) {
            break;
        }
        /* Update the read offset for next request */
        startingLcn.StartingLcn.QuadPart += bytesReturned*8;
    }

    CloseHandle(hFile);
	CloseHandle(hVolume);
    free (volBitmap);

    return 0;
}

Windows P2V machine’s boot error status: 0xC000000E

Q. I’m working on developing a P2V solution for Windows machines. For one client I got the boot error: 0xC000000E (Info: The boot selection failed because a required device is inaccessible”). How to fix the boot error 0xC000000E for a Virtual Machine?

Ans:
If this problem happens for physical machine check below link:
http://www.prime-expert.com/articles/b16/fix-0xC000000E-required-device-is-inaccessible.php

The error looks like this:

boot_error_status_0xC000000E

The explanation of the error is given nicely at:

http://www.prime-expert.com/articles/b16/fix-0xC000000E-required-device-is-inaccessible.php

Process to fix the corrupt BCD:

The tool used for fixing this is: BCDboot.
1. Mount the virtual hard disk VHD files on the Windows which contains Boot & System partition. (Boot & System partition can be on two different disk.) To know more about drive types check: http://support.microsoft.com/kb/314470
2. Lets assume your Boot partition is mounted at B:\ and System partition is mounted at S:\
3. Then use below commands to fix the corrupt BCD.

Save original BCD data

 cd S:\Boot
 ren BCD BCD.old

Run BCDboot to fix corrupt BCD:

This will fix both UEFI & BIOS based booting.

 BCDboot.exe B:\Windows /s S: /f ALL

If your HyperVisor doesn’t support UEFI booting use below command:

 BCDboot.exe B:\Windows /s S:

This will fix your 0xC000000E boot problem.

Windows Random File Generator

Q. Generate random files in Windows. Let the random files / directories generated have compression or encryption attribute set randomly. Also the files as well as directories should have the alternate data streams (ADS) associated with. There should be provision of updating certain parts of this randomly generated data. This kind of utility is very useful to test file system backup products.

Ans: For long time I was searching for the Unix like “dd” command line utility for Windows.  I came across this utility: Random Data File Creator (RDFC). This gives us “dd” kind of functionality. Download this utility and keep it in your C:\

Use below batch script to generate random files and directories with compression / encryption attributes and alternate data streams associated with them.

echo off
set testname=%0
set foldername=%1
set fext=%2
set size=%3
set unit=%4
set no=%5
set outerfolder=%6
set innerfolder=%7
set /A filenumber = 1

set dest=%foldername%
set str=%dest: =%
mkdir %str%
set /A a=0
:Loop1
	set dest1=%str%\%a%
	set str1=%dest1: =%
	mkdir %str1%
	set /A b=0
	:Loop2
		set dest2=%str1%\%b%
		set str2=%dest2: =%
		mkdir %str2%
		set /A c=0
		:Loop3
			set fname=%str2%\file-%filenumber%.%fext%
			set tmpfname=%str2%\file_tmp
			c:\rdfc.exe %tmpfname% %size% %unit% overwrite
			type nul >> %fname%
			:: append data to existing file
			type %tmpfname% >> %fname%
			:: create alternate data stream
			set /A adsproperty = %filenumber% %% 2
			IF %adsproperty%==1 (type %tmpfname% >> %fname%:my_file_ads)
			del %tmpfname%
			set /A filenumber = %filenumber% + 1
			set /A c=%c%+1
			IF %c%==%no% (goto end3) ELSE (goto loop3)
		:end3
		rem *******************
		:: set compression / encryption attribute
		set /A dirproperty = %b% %% 3
		IF %dirproperty%==1 (compact /C /S:%str2%)
		IF %dirproperty%==2 (cipher /E /S:%str2%)
		set tmpfname=%str1%\file_tmp
		c:\rdfc.exe %tmpfname% %size% %unit% overwrite
		set /A adsproperty = %b% %% 2
		IF %adsproperty%==1 (type %tmpfname% >> %str2%:my_dir_ads)
		del %tmpfname%
		set /A b=%b%+1
		IF %b%==%innerfolder% (goto end2) ELSE (goto loop2)
	:end2
	rem *******************
	set /A a=%a%+1
	IF %a%==%outerfolder% (goto end1) ELSE (goto loop1)
:end1

:: usage random_files.bat foldername EXTENTION SIZE UNIT[B|kB|MB|GB] NO_OF_FILES no_outerfolders no_innerfolders

Usage:

C:\>random_files.bat D:\test bin 1 kB 2 3 3

This will generate D:\test folder. Inside this folder there will be 3 subfolders created numbered: 0, 1, 2. Inside each of these folders again there will be three subfolders numbered 0, 1, 2. Inside each of these folders there will be 2 random 1 KB files with .bin extension are created. Starting from the folder named 1, each of the third inner folder and its contents will be compressed. Starting from inner folder numbered 2, each of the third folder and its contents will be encrypted. Each of the second file created will have an alternate data stream (ADS) associated with it. Each of the second inner folder created will have alternate data stream (ADS) associated with it.

C:\>random_files.bat D:\test bin 512 B 4 3 1

Running this command for the second time only updates first two files in the 0’th inner folder in all the three outer folders. Now those first two files will have size 1.5 kB. (The new data is appended). Apart from this two new files of size 512 bytes are also created in each of the 0’th inner folder.

You can have any permutation to generate random data and keep on updating that data with any combination 🙂

 

Windows how to find disk number of mounted VHD?

Q. Once you mount a VHD on Windows, how to find the disk number ‘n’ of the mounted vhd? This is required to access the mounted VHD in the disk form as \\.\PhyiscalDriven.

Ans:

Use below diskpart command, it lists all the mounted VHDs and their corresponding disk numbers.

DISKPART> list vdisk
VDisk ### Disk ### State Type File
 --------- -------- -------------------- --------- ----
 VDisk 0 Disk 2 Attached not open Expandable S:\UBMD.vhd
 VDisk 1 Disk 1 Attached not open Expandable C:\resOCB\test_0.vhd

From the example S:\UBMD.vhd is mounted at disk number 2 and C:\resOCB\test_0.vhd is mounted at disk number 1.

Windows how to find the boot disk?

Q. On a Windows server how to find which physical disk is boot disk?

Ans: diskpart.exe helps to identify which is the boot disk. Use following commands:

C:\Users\Administrator>diskpart
Microsoft DiskPart version 6.1.7601
 Copyright (C) 1999-2008 Microsoft Corporation.
 On computer: SYSTEMS11
DISKPART> list disk
  Disk ###  Status         Size     Free     Dyn  Gpt
   --------  -------------  -------  -------  ---  ---
   Disk 0    Online          465 GB  1024 KB
   Disk 1    Online          100 GB  2048 KB
   Disk 2    Online         7633 MB      0 B
DISKPART> select disk 0
Disk 0 is now the selected disk.
DISKPART> detail disk
WDC WD5000AAKX-001CA0 ATA Device
 Disk ID: 84BF18DB
 Type   : SATA
 Status : Online
 Path   : 0
 Target : 0
 LUN ID : 0
 Location Path : PCIROOT(0)#PCI(1F02)#ATA(C00T00L00)
 Current Read-only State : No
 Read-only  : No
 Boot Disk  : Yes
 Pagefile Disk  : Yes
 Hibernation File Disk  : No
 Crashdump Disk  : Yes
 Clustered Disk  : No
  Volume ###  Ltr  Label        Fs     Type        Size     Status     Info
   ----------  ---  -----------  -----  ----------  -------  ---------  --------
   Volume 0                      NTFS   Partition    100 MB  Healthy    System
   Volume 1     C                NTFS   Partition    165 GB  Healthy    Boot
   Volume 2     S   VM           NTFS   Partition    299 GB  Healthy

The “detail disk” command outputs below parameter which identifies whether the selected disk is boot disk or not:
Boot Disk  : Yes

The disk selected in example was the boot disk, if it was not the case the output for this parameter would have been:
Boot Disk  : No

On Windows XP I observed that diskpart’s “detail disk” has no “Boot Disk” entry in the output. In that case we can check the list of volumes displayed as output of “detail disk”. The disk which contains the volumes which “Boot” volume is the boot disk.

Windows how to get type (IDE/ATA/SATA/SCSI/USB/Virtual) of physical disk?

Q. On Windows server given the physical disk \\.\PHYSICALDRIVEn identify whether the disk is ATA, SATA. SCSI disk, USB disk or it is a mounted VHD file.

Ans:
There is no straightforward way in C++ to do this. Windows provide SetupDiEnumDeviceInterfaces APIs to get the properties of devices. But I found it bit complex to solve the problem mentioned.

On the other hand diskpart.exe provides simple interface to identify the type of the disk. Use following commands to get the type of disk:

C:\Users\Administrator>diskpart
Microsoft DiskPart version 6.1.7601
 Copyright (C) 1999-2008 Microsoft Corporation.
 On computer: SYSTEMS11
DISKPART> list disk
  Disk ###  Status         Size     Free     Dyn  Gpt
   --------  -------------  -------  -------  ---  ---
   Disk 0    Online          465 GB  1024 KB
   Disk 1    Online          100 GB  2048 KB
   Disk 2    Online         7633 MB      0 B
DISKPART> select disk 0
Disk 0 is now the selected disk.
DISKPART> detail disk
WDC WD5000AAKX-001CA0 ATA Device
 Disk ID: 84BF18DB
 Type   : SATA
 Status : Online
 Path   : 0
 Target : 0
 LUN ID : 0
 Location Path : PCIROOT(0)#PCI(1F02)#ATA(C00T00L00)
 Current Read-only State : No
 Read-only  : No
 Boot Disk  : Yes
 Pagefile Disk  : Yes
 Hibernation File Disk  : No
 Crashdump Disk  : Yes
 Clustered Disk  : No
  Volume ###  Ltr  Label        Fs     Type        Size     Status     Info
   ----------  ---  -----------  -----  ----------  -------  ---------  --------
   Volume 0                      NTFS   Partition    100 MB  Healthy    System
   Volume 1     C                NTFS   Partition    165 GB  Healthy    Boot
   Volume 2     S   VM           NTFS   Partition    299 GB  Healthy
DISKPART>

In the information provided by “detail disk” check the entry:
Type   : SATA 

If the physical disk is ATA drive the type entry in “detail disk” appear as:
Type   : ATA

If the physical disk is SCSI drive the type entry in “detail disk” appear as:
Type   : SCSI

If the physical disk is USB drive the type entry in “detail disk” appear as:
Type   : USB

If the physical disk is mounted VHD/VHDX file the type entry in “detail disk” appear as:
Type   : Virtual