Python Sample Exploitation ETL

This sample shows the steps needed to implement a file exploitation ETL in Truxton.

Sample File Format

This sample will exploit a fake file format we call Acme. Acme Corporation is a known supplier of nefarious devices and explosives. Their file format begins with a five byte magic value followed by eleven bytes in a data structure. If the sixth byte in the file is 0x11 then it is a serial number file that uniquely identifies the user.

0000h: 88 77 66 55 00 11 22 33 44 55 66 77 88 99 AA BB
0010h: CC

Source Code

 1 import truxton
 2 
 3 def main():
 4 
 5   etl = truxton.etl()
 6   etl.name = "Acme Exploitation"
 7   etl.description = "This exploits Acme Corporation data files"
 8   etl.queue = "wiley"
 9   etl.stage = 40
10   etl.addtype(11000)
11 
12   message = etl.getmessage()
13 
14   while message is not None:
15     file_in_truxton = message.file()
16 
17     file_in_truxton.seek(5)
18 
19     next_byte = file_in_truxton.read(1)
20     if next_byte[0] == 0x11:
21       # Serial Number. The next 8 bytes are a serial number
22       file_in_truxton.seek(6)
23       serial_number = file_in_truxton.read(8)
24 
25       artifact = file_in_truxton.newartifact()
26       artifact.type = truxton.ENTITY_TYPE_SERIAL_NUMBER
27       artifact.value = serial_number.hex()
28       artifact.datatype = truxton.DATA_TYPE_uint8_t
29       artifact.offset = 6
30       artifact.length = 8
31       artifact.save()
32 
33     message = etl.getmessage()
34 
35 if __name__ == "__main__":
36     main()

Code Walkthrough

Lines 5-10 setup the ETL. The message queue name will be "acme", we are an early stage and want to receive Acme files (11000 was chosen as the identifier of Acme files).

Line 12 starts the ETL logic and waits until a message arrives on the "wiley" queue.

Line 15 opens the file so we can read from it.

Lines 19-20 read the fifth byte in the file and checks it for validity.

Lines 25-31 creates an artifact (which will be stored in the Entity table in the database) and saves it to Truxton.

Line 26 sets the type of artifact to a serial number. This allows analysts to quickly find items of interest by their type.

Line 28 stores the format of how the serial number was stored in the file.

Line 31 saves the data to Truxton. It will create a record in the Entity table in the database. Saving the artifact will cause Truxton to route it to any ETL that has subscribed to Type_Artifact messages.

Development and Debugging

Truxton ETLs assume they are part of a processing stream instead of processing a file from the local system. This can slow your development cycle down. Here's one strategy that will make your development iterations quicker:

1. Perform a load with your sample file in it
2. Find the file using the desktop GUI, copy the file identifier.
3. Stop the Truxton service
4. Modify your program to use the sendme() function.
5. Now every time you run your program you will immediately receive that file to play with.

 1 import truxton
 2 
 3 def main():
 4 
 5   etl = truxton.etl()
 6   etl.name = "Acme Exploitation"
 7   etl.description = "This exploits Acme Corporation data files"
 8   etl.queue = "wiley"
 9   etl.stage = 40
10   etl.addtype(11000)
11 
12   message = etl.getmessage()
13 
14   while message is not None:
15     file_in_truxton = message.file()
16 
17     file_in_truxton.seek(5)
18 
19     next_byte = file_in_truxton.read(1)
20     if next_byte[0] == 0x11:
21       # Serial Number. The next 8 bytes are a serial number
22       file_in_truxton.seek(6)
23       serial_number = file_in_truxton.read(8)
24 
25       artifact = file_in_truxton.newartifact()
26       artifact.type = truxton.ENTITY_TYPE_SERIAL_NUMBER
27       artifact.value = serial_number.hex()
28       artifact.datatype = truxton.DATA_TYPE_uint8_t
29       artifact.offset = 6
30       artifact.length = 8
31       artifact.save()
32 
33     message = etl.getmessage()
34 
35 if __name__ == "__main__":
36     main()