IP address validation

Q. How to validate an IP address (IPv4 or IPv6), given as a string input ?

Ans: Using regex is not good idea. Rather use function inet_pton to do the validation task. The pseudo code for IP address validation is as below:

bool
isValidIPAddress(
    IN char *ipAddr
    )
{
    unsigned char buf[sizeof(struct in6_addr)];
    int ret;

    ret = inet_pton(AF_INET, ipAddr, buf);
    if (1 == ret) {
        return true;
    }

    ret = inet_pton(AF_INET6, ipAddr, buf);
    if (1 == ret) {
        return true;
    }
    return false;
}

Advertisements

How to close socket after tiny inactivity timeout?

Q. There is TCP connection, it needs to be closed and re initiated if there is no activity for a timeout period. The timeout period is tiny, of the order of few seconds or minutes?

Ans:
It is not easy to identify a dead or non responsive peer in TCP connection. Normally when the timeout period is long of the order of few hours, you can solve the problem using the TCP keepalive. But it makes no sense to use TCP keepalive when timeout period is tiny, of the order of few seconds. This tiny timeouts can be handled by select system call to check if there is any activity on the socket within the timeout period.

Lets say you’re listening on socket and you expect to read certain packet of known size at least once every 40 seconds, in case you don’t receive the packet you expect to close the TCP connection. You can solve this with following pseudo code.

int
readSocket(
    int fdSocket,
    char *buffer,
    int length
    )
{
    int totalRead;
    int thisTimeRead;
    fd_set readSet;
    int result;
    struct timeval timeout;

    for (totalRead = 0; totalRead < length; ) {
        FD_ZERO(&readSet);
        FD_SET(fdSocket, &readSet);
        timeout.tv_sec= 40;
        timeout.tv_usec = 0;
        /* wait till there is data to be read on the socket or timeout happens */
        result = select(fdSocket+1, &readSet, NULL, NULL, &timeout);
        if (-1 == result) {
            /* Some error in select api */
            if (EINTR == errno) {
                continue;
            }
            /* close socket and return error */
            close(fdSocket);
            return -1;
        }
        /* check if timeout happened */
        if (0 == result) {
            /* timeout, close the socket return error */
            close(fdSocket);
            return -1;
        }

        thisTimeRead = read(fdSocket, buffer+totalRead, length-totalRead);
        if (thisTimeRead < 0) {
            if (EINTR == errno) {
                continue;
            }
            return -1;
        } else if (0 == thisTimeRead) {
            /* EOF detected */
            break;
        }
        /* data is read from the socket, make the connection status active */
        totalRead += thisTimeRead;
    }

    return totalRead;
}

The preferred way to achieve the same effect for both socket read and write operations is by setting socket options: SO_RCVTIMEO and SO_SNDTIMEO

http://linux.die.net/man/7/socket

SO_RCVTIMEO and SO_SNDTIMEO
Specify the receiving or sending timeouts until reporting an error. The argument is a struct timeval. If an input or output function blocks for this period of time, and data has been sent or received, the return value of that function will be the amount of data transferred; if no data has been transferred and the timeout has been reached then -1 is returned with errno set to EAGAIN or EWOULDBLOCK, or EINPROGRESS (for connect(2)) just as if the socket was specified to be nonblocking. If the timeout is set to zero (the default) then the operation will never timeout. Timeouts only have effect for system calls that perform socket I/O (e.g., read(2), recvmsg(2), send(2), sendmsg(2)); timeouts have no effect for select(2), poll(2), epoll_wait(2), and so on.

How to parse a TAR file?

Q. From a tar file how to list it’s contents in the form of filename and their sizes?

Ans:

Details of the tar file format can be found here: http://en.wikipedia.org/wiki/Tar_%28computing%29

Original Tar file format had two problems:
1. archiving file of size greater than 8 GB
2. archiving file whose name is > 100 bytes.

Below is the tar header format:

{
    char name[100];
    char mode[8];
    char uid[8];
    char gid[8];
    unsigned char size[12];
    char mtime[12];
    char chksum[8];
    char typeflag;
    char linkname[100];
    char magic[6];
    char version[2];
    char uname[32];
    char gname[32];
    char devmajor[8];
    char devminor[8];
    char prefix[155];
    char pad[12];
}

These problems later got fixed in GNU tar format.

a. For parsing file size use below rules:
Numeric values are encoded in octal numbers using ASCII digits, with leading zeroes. For historical reasons, a final NUL or space character should be used. Thus although there are 12 bytes reserved for storing the file size, only 11 octal digits can be stored. This gives a maximum file size of 8 gigabytes on archived files. To overcome this limitation, star in 2001 introduced a base-256 coding that is indicated by setting the high-order bit of the leftmost byte of a numeric field. GNU-tar and BSD-tar followed this idea.

b. For parsing file name use below rules:
If the first character of prefix is \0 (null character), the file name is name; otherwise, it is prefix/name. Files whose pathnames don’t fit in that length can not be stored in a tar archive.

c. LongLink rule:
I found that though rule ‘b’ is documented it is not followed. Large filenames are stored in tar using LongLink concept. It is special tar entry used to store only large filenames. The LongLink type header has typeflag ‘L’. The file name for this header is “././@LongLink“. The data content for this file are nothing but the “long” filename for the next archive entry.

To generate a “gnu” format tar use below command in shell:

$ tar --format=gnu -cvf xyz.tar dir

Following code parse a “gnu” tar file and print the file name & file size of all the archived files.

struct GnuTarHeader
{
    char name[100];
    char mode[8];
    char uid[8];
    char gid[8];
    unsigned char size[12];
    char mtime[12];
    char chksum[8];
    char typeflag;
    char linkname[100];
    char magic[6];
    char version[2];
    char uname[32];
    char gname[32];
    char devmajor[8];
    char devminor[8];
    char prefix[155];
    char pad[12];
};
 
void validateTarHeader(GnuTarHeader *tarHeader);
void parseFileSize(GnuTarHeader *tarHeader);
void parseFileName(GnuTarHeader *tarHeader);
void parseLongLink(GnuTarHeader *tarHeader, int fd);
void parseTarHeader(GnuTarHeader *tarHeader);
 
std::string currentFileName;
unsigned long long currentFileSize;
bool lastLongLinkHeader;
 
char TAR_MAGIC[] = "ustar ";
 
int main(int argc, char **argv)
{
    int fd;
    int ret;
    unsigned long long seek;
    GnuTarHeader gnuHeader, emptyHeader;
    int emptyHeaders = 0;
 
    if (argc != 2) {
        printf ("Usage: %s tar_file_name\n", argv[0]);
        return 1;
    }
 
    fd = open(argv[1], O_RDONLY);
    assert (fd != -1);
 
    memset (&emptyHeader, 0, 512);
 
    while (1) {
        ret = read(fd, &gnuHeader, 512);
        assert(ret == 512);
        if (0 == memcmp(&gnuHeader, &emptyHeader, 512)) {
            emptyHeaders ++;
            if (2 == emptyHeaders) {
                break;
            }
            continue;
        }
        emptyHeaders = 0;
        validateTarHeader(&gnuHeader);
        if ('L' == gnuHeader.typeflag) {
            parseLongLink(&gnuHeader, fd);
        } else {
            parseTarHeader(&gnuHeader);
            seek = (currentFileSize/512) + (currentFileSize%512 ? 1 : 0);
            seek *= 512;
            seek = lseek(fd, seek, SEEK_CUR);
            assert(seek != -1);
        }
    }
 
    return 0;
}
 
void validateTarHeader(GnuTarHeader *tarHeader)
{
    for (int i=0; i<6; i++) {
        assert(tarHeader->magic[i] == TAR_MAGIC[i]);
    }
}
 
void parseFileSize(GnuTarHeader *tarHeader)
{
    int i;
 
    // parse the file size.
    currentFileSize = 0;
 
    if (tarHeader->size[0] & (0X01 << 7)) {
        // file size > 8 GB.
        for (i=1; i<12; i++) {
            currentFileSize *= 256;
            currentFileSize += tarHeader->size[i];
        }
    } else {
        // file size < 8 GB.
        for (i=0; i<12; i++) {
            if ((0 == tarHeader->size[i]) || (' ' == tarHeader->size[i])) {
                continue;
            }
            currentFileSize *= 8;
            currentFileSize += (tarHeader->size[i] - '0');
        }
    }
}
 
void parseFileName(GnuTarHeader *tarHeader)
{
    int i;
    char fileName[256];
 
    currentFileName = "";
 
    if (0 != tarHeader->prefix[0]) {
        for (i=0; i<155; i++) {
            if (0 == tarHeader->prefix[i]) {
                break;
            }
            fileName[i] = tarHeader->prefix[i];
        }
        fileName[i] = '\0';
        currentFileName = fileName;
        currentFileName += "//";
    }
 
    for (i=0; i<100; i++) {
        if (0 == tarHeader->name[0]) {
            break;
        }
        fileName[i] = tarHeader->name[i];
    }
 
    fileName[i] = '\0';
    currentFileName += fileName;
}
 
void parseLongLink(GnuTarHeader *tarHeader, int fd)
{
    int ret;
    char fileName[512+1]; // last byte for '\0''
 
    currentFileName = "";
    parseFileSize(tarHeader);
    while (true) {
        ret = read (fd, fileName, 512);
        if (currentFileSize > 512) {
            fileName[512] = '\0';
        } else {
            fileName[currentFileSize] = '\0';
            currentFileName += fileName;
            break;
        }
        currentFileSize -= 512;
        currentFileName += fileName;
    }
 
    lastLongLinkHeader = true;
}
 
void parseTarHeader(GnuTarHeader *tarHeader)
{
    parseFileSize(tarHeader);
 
    // parse the filename.
    if (false == lastLongLinkHeader) {
        parseFileName(tarHeader);
    }
 
    lastLongLinkHeader = false;
    printf ("%s %llu\n", currentFileName.c_str(), currentFileSize);
}

How to handle signal SIGPIPE?

Q. C program on Unix crashed after it attempted to write on a closed socket. How to handle this?

Ans:
In Unix environment, when a process attempts to write on a socket which is shutdown for writing or which is not connected (anymore), OS sends signal SIGPIPE to that process.
Default behavior of the signal SIGPIPE is to terminate the process. To avoid this your process must ignore the signal SIGPIPE. In that case if your process attempts to write on a closed socket, the send() / write() call returns -1 and errno is set to EPIPE. Your process should handle this failure in the write accordingly. This is how you can stop abruptly crashing of your program.

Below C code shows how to ignore signal SIGPIPE.

{
    int ret;
    struct sigaction sa;

    /*
     * There are chances that our application sends 
     * data to a closed socket. This generates SIGPIPE
     * signal, and results into process termination.
     * We must ignore this signal, as our application is 
     * equipped to handle errors happening on socket writes.
     */

    sa.sa_handler = SIG_IGN;
    sa.sa_flags = 0;
    ret = sigaction(SIGPIPE, &sa, NULL);
    assert(-1 != ret);
}

GDB: Get stack trace of all threads of running process

Q. How to get the stack trace of a running c/c++ process on Unix.

Ans:

1. Attach GDB to a running process:

Get the pid of running process foo. $ps -aef | grep foo
Attach gdb to the pid of process foo
$sudo gdb -p 21884

2. Save the stack trace of all running threads in a log file.

(gdb) set pagination off
(gdb) set logging on
 Copying output to gdb.txt.
(gdb) thread apply all bt

3. Detach gdb from running process

(gdb) detach
 Detaching from program: foo, process 21884

After this you’ll find the stack trace of all the threads of your running process in file text gdb.txt This is extremely useful to debug the hung process.

GDB: print the c++ STL object

For me debugging becomes painful in Unix compared to Windows.

I came across an interesting problem. I had to display the contents of unordered_set of a running process on Ubuntu server. I attached the GDB to running process and tried to display the contents. Here is what I got:

1. Attach GDB to a running process:

Get the pid of running process foo. $ps -aef | grep foo Attach gdb to the pid of process foo $sudo gdb -p pid

2. Display the unordered_set contents

(gdb) print set_contents
$1 = {<std::tr1::__unordered_set<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::tr1::hash<std::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::equal_to<std::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::basic_string<char, std::char_traits<char>, std::allocator<char> > >, false>> = {<std::tr1::_Hashtable<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::_Identity<std::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::equal_to<std::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::tr1::hash<std::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::tr1::__detail::_Mod_range_hashing, std::tr1::__detail::_Default_ranged_hash, std::tr1::__detail::_Prime_rehash_policy, false, true, true>> = {<std::tr1::__detail::_Rehash_base<std::tr1::__detail::_Prime_rehash_policy, std::tr1::_Hashtable<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::_Identity<std::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::equal_to<std::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::tr1::hash<std::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::tr1::__detail::_Mod_range_hashing, std::tr1::__detail::_Default_ranged_hash, std::tr1::__detail::_Prime_rehash_policy, false, true, true> >> = {<No data fields>}, <std::tr1::__detail::_Hash_code_base<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::_Identity<std::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::equal_to<std::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::tr1::hash<std::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::tr1::__detail::_Mod_range_hashing, std::tr1::__detail::_Default_ranged_hash, false>> = {
        _M_extract = {<std::unary_function<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::basic_string<char, std::char_traits<char>, std::allocator<char> > >> = {<No data fields>}, <No data fields>},
        _M_eq = {<std::binary_function<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool>> = {<No data fields>}, <No data fields>},
        _M_h1 = {<std::unary_function<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, unsigned long>> = {<No data fields>}, <No data fields>},
        _M_h2 = {<No data fields>}}, <std::tr1::__detail::_Map_base<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::_Identity<std::basic_string<char, std::char_traits<char>, std::allocator<char> > >, true, std::tr1::_Hashtable<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::_Identity<std::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::equal_to<std::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::tr1::hash<std::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::tr1::__detail::_Mod_range_hashing, std::tr1::__detail::_Default_ranged_hash, std::tr1::__detail::_Prime_rehash_policy, false, true, true> >> = {<No data fields>},
      _M_node_allocator = {<__gnu_cxx::new_allocator<std::tr1::__detail::_Hash_node<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, false> >> = {<No data fields>}, <No data fields>}, _M_buckets = 0x1fc6550, _M_bucket_count = 11, _M_element_count = 1, _M_rehash_policy = {
        _M_max_load_factor = 1, _M_growth_factor = 2,
        _M_next_resize = 11}}, <No data fields>}, <No data fields>}

There is no way you can make out what are the contents of STL object with this output.

Then I came across this article: https://sourceware.org/gdb/wiki/STLSupport

Following steps helped me to achieve my goal:

1. Check-out the latest Python libstdc++ printers to a place on your machine. In a local directory, do:
svn co svn://gcc.gnu.org/svn/gcc/trunk/libstdc++-v3/python

2. Add the following to your ~/.gdbinit. The path needs to match where the python module above was checked-out. So if checked out to: /home/maude/gdb_printers/, the path would be as written in the example:

python
import sys
sys.path.insert(0, '/home/maude/gdb_printers/python')
from libstdcxx.v6.printers import register_libstdcxx_printers
register_libstdcxx_printers (None)
end

After this GDB started displaying the contents of unordered_set as expected:

(gdb) print set_contents 
$1 = std::unordered_set with 1 elements = {"2014-03-15/449782CE-10E6-4233-A143-61CB6314E103" }

Multithreading: Errno 9 (EBADF) – Bad File Number

If you’re randomly hitting “Bad File Number” error (Errno: 9) in a multithreaded application on Unix, then most likely you’re trapped in “Opened once but closed twice” bug.

What is this “Opened once but closed twice” bug?

Check out the below code:

foo ()
 {
     fd = open();
     // some processing.
     close (fd);
     // some processing.
     close (fd);
 }

The above code will work smoothly for a single threaded application. But in case of multithreading, if a thread “X” call open() (either in same function foo or any other function bar), when thread “Y” is in between the two close() calls, then in that case, thread “X” will get the same fd for open which thread “Y” was using (since open always return the smallest unused descriptor). File descriptors are shared by all the threads. So when thread “Y” carries out its second close(), it actually closes the file descriptor of thread “X”, which was valid & in use.

FD numbers are shared by all the threads of the same process. That means when thread “Y” closes FD number “n”, twice. Then in the second case of closing descriptor “n”, can result closing in use descriptor “n” by thread “X”. When the thread “X” starts using the descriptor “n”, it gets error – “Bad File Number” (Errno: 9).

Thus closing a descriptor twice in multithreaded application leads to random “Bad File Number” error (Errno: 9).