TGUID

A TGUID (Truxton GUID) is a 128-bit identifier used throughout Truxton. It can be stored and interoperate with a GUID data type. However, it is not a GUID. It does not adhere to any specification at all. It is merely the same size as a GUID but easier to debug.

In Memory Layout

A TGUID differs from GUID in that the string/logical value of a TGUID is the same as the series of bytes in memory. Think of it as a 128-bit big-endian integer in RAM. Consider the following bytes as they appear in RAM:

0x00, 0x11, 0x22, 0x33, 0x44, 0x55, 0x66, 0x77, 0x88, 0x99, 0xAA, 0xBB, 0xCC, 0xDD, 0xEE, 0xFF

Converting those bytes to a TGUID will yield: {00112233-4455-6677-8899-AABBCCDDEEFF}

Converting those same bytes to a GUID will yield: {33221100-5544-7766-8899-AABBCCDDEEFF}

Types of TGUIDs

Truxton takes a very liberal interpretation of "globally unique identifier." Truxton's view of a TGUID is "128 bits that I can do whatever I want with."

File ID

A file identifier in Truxton can be totally random but it usually generated by the loader or an ETL process. In that case, the identifier is made up of a time, random and a counter value. This satisfies a couple of requirements we have of file identifiers (other than being globally unique):

• Quickly Created. Generating purely random data is a lengthy process when you are generating millions of them.
• Make database partitioning possible. By having part of the identifier being a timestamp, we can partition the File table by algorithm.

Let's take apart a sample file identifier:

5da98fe7-14e6-f663-7143-a0e80000000a

File ID Fields

	Field A	Field B	Field C
Value	5da98fe7	14e6-f663-7143-a0e8	0000000a
Type	Time	Random	Counter
Meaning	2019-10-18T10:11:51	Random	ID number 10

Field A

The first field is a big endian representation of a Unix timestamp. 5da98fe7 is 1,571,393,511 seconds since January 1, 1970 or October 18, 2019 10:11:51 AM. This makes it possible to construct a query to find out what was loaded on a particular date. To see load activity on October 18, 2019:

SELECT * FROM "File" WHERE "ID" BETWEEN '5da90080-0000-0000-0000-000000000000'::uuid AND '5da9b74f-ffff-ffff-ffff-ffffffff'::uuid

Field B

The second field is sixty four bits of random data. This is generated using the secure random number generator function rand_s which uses the RtlGenRandom.

Field C

The last field is a thirty two bit counter value. Every time an identifier is generated, this field is incremented. In our example, we can tell this is the tenth identifier that was generated.

Media ID

The media ID in Truxton is based on time. A clock tick is six hours. The epoch is 2016-11-01. The rollover will happen September 18, 2061.

The first two bytes of the identifier is the number of clock ticks since 2016-11-01. The remaining 30 bytes are random using the cryptographicaly secure random number generator in Windows.

Example: 10e7b286-4f9f-3e2e-10da-07d5db37944c

The time field is 10e7

Having a Media ID that can be sorted on time allows for the partitioning of data in the database. You can partition your data by year if you want to.

ETL ID

The job of an ETL ID is to identify an ETL on the network. ETL IDs are used in status messages when reporting to Les.

Fields

The format of an ETL ID is as follows:

SSIIRRRR-PPPP-PPTT-TTTT-AAAAAAAAAAAA

Field	Meaning
SS	The stage of the ETL.
II	The instance of the ETL on the machine running the process.
RRRR	A random value.
PPPPPP	The process ID of the ETL process.
TTTTTT	The thread ID of the ETL process. A single process can host multiple ETLs.
AAAAAAAAAAAA	The IP address of the machine running the ETL process.

Example: 03015678-0079-C400-7514-192168001014

Field	Value	Meaning
SS	03	Stage 3
II	01	First instance of the ETL
RRRR	5678	A random value
PPPPPP	0079C4	Process ID 31172
TTTTTT	007514	The thread ID 29972
AAAAAAAAAAAA	192168001014	The IP address is 192.168.1.14

Using this information, you can attach a debugger to the machine using the IP address and debug the process using the process id and thread id. From our example:

C:\> mydbg /ip:192.168.1.14 /pid:31172

Customer ID

Customer identifiers are proprietary.

API Considerations

Internally, Truxton uses TGUIDs. The external API intended for use by customers, uses GUIDs as trying to explain this to them would lead to never ending confusion. It is far better if the Truxton libraries keep TGUIDs to themselves. Like a family secret.