[Note: This data is terribly out of date, based on the peliminary design for
 Deadwood 2 back in 2007.  In 2010, in order to implement full recursion,
 I added a whole bunch of types, such as CNAME referrals, NS referrals,
 and a couple of types for truncated replies.  I also have two different
 types for "not found" packets; one for NXDOMAIN, and another for 
 non-NXDOMAIN "not found" replies.  See DwRecurse.h and DwRecurse.c for
 an overview; there are extensive comments that explain the datatypes
 stored here.

 I should also point out the simple answers have offsets to RRs in the
 answers, and are no longer stored in compressed form.  The following data
 types are only used in the "tiny" 2.3 branch of Deadwood]

Cache data: Cached data will use the hash (see HASH.DESIGN) 
            and be stored like this:

       <raw DNS answer><ANCOUNT><NSCOUNT><ARCOUNT><type>

       The raw DNS answer is the packet, as-is sent from the server (complete
       with compression pointers, etc), starting after the end of the 
       question.  Basically, we reject DNS replies that don't have a QDCOUNT
       of 1, that don't have the expected DNS question, or that don't have
       the expected query ID.  We then shave off the DNS header (noting
       ANCOUNT, NSCOUNT, and ARCOUNT), the question, and put the rest of
       the reply in the cache, adding (AN/NS/AR)COUNT at the end of the
       cached packet.

       The cache is an assosciative array; the "key" is the DNS question;
       the value is the answer as described above.

       When getting an answer from the cache, we sew the key, value, and 
       current query to give the client an answer.  Basically, in the
       header:

       ID: As sent by the client
       QR: 1, since it is a reply
       OPCODE: 0
       AA: 0 (since we're a cache)
       TC: 0 (if we get a TC of 1 from the remote server, we do not
              cache the reply, but pass on a reply to the client with
              the TC bit on.  This way, they can use DwTcp to get
              a non-cached answer.  Yes, the real solution is to use
              TCP and UDP in the same daemon, but we'll deal with that 
              later)
       RD: 1 (We reject queries where this is 0)
       RA: 1 (Some DNS stub resolvers actually check this bit)
       Z: 0
       RCODE: 0 or 3 if "type" in cache is 1 (would be 2 if we have a 
              server failure)
       QDCOUNT: 1
       (AN/NS/AR)COUNT: As per the last 6 bytes of the value in the cache
       Question: As per the key + (0x0001) (CLASS 1)
       Answer: The rest of the value after shaving off (AN/NS/AR)COUNT

type: Type is an 8-bit value placed at the end of the string for each
      cache entry.  Type currently has two values:

	0: This indicates that the rest of the string is stored as described
           here, with an RCODE of 0

	1: This indicates that the rest of the string is stored as described
           here, with an RCODE of 1

== The cache on disk ==

The cache file on disk is pretty simple; it just has a 4-byte header, 
followed by the maximum number of elements the cache can have (not really
used in Deadwood 2.4), followed by each hash element.

Each hash element has three values: A 4-byte big endian number storing the
number of bytes in the key, followed by the actual key.  The same string
format for the value in the hash.  This is followed by an 8-byte timestamp
that uses a Deadwood-specific format for the time when the entry expires.

The expires are stored with the least important entries being first in
the file, followed by more and more important entires.

OK, in more detail, there are three data types in the cache:

1) 4-byte big endian binary numbers

2) Strings (this is a black box, as far as the code that writes to disk is
   concerned)

3) 8-byte binary timestamps.

The cache starts with a 4-byte header '\0'DX2 (An ASCII null, followed by 
"DX2" in ASCII).  It then has a 4-byte number with the maximum allowed number 
of entries in the cache.

This is followed by each hash element.

4-byte: Length of key
String: Key
4-byte: Length of value
String: Value
Timestamp: Expire time for record

The records closer to the end of the file are considered more important
than records at the beginning of the file; this means they are less likely
to be axed when the hash is full and needs to remove elements.



