Difference between revisions of "Python Sample Exploitation ETL"
| Line 62: | Line 62: | ||
Lines 19-20 read the fifth byte in the file and checks it for validity. | Lines 19-20 read the fifth byte in the file and checks it for validity. | ||
| − | Lines 25-31 creates an artifact (which will be stored in the <code>[[Entity Table | Entity]]</code> table in the database) and saves it to Truxton. | + | Lines 25-31 creates an artifact (which will be stored in the <code><nowiki>[</nowiki>[[Entity Table|Entity]]<nowiki>]</nowiki></code> table in the database) and saves it to Truxton. |
| − | Line 26 sets the [[Entity Types | type of artifact]] to a serial number. | + | Line 26 sets the [[Entity Types|type of artifact]] to a serial number. |
This allows analysts to quickly find items of interest by their type. | This allows analysts to quickly find items of interest by their type. | ||
| − | Line 28 stores the [[DATA_TYPE | format]] of how the serial number was stored in the file. | + | Line 28 stores the [[DATA_TYPE|format]] of how the serial number was stored in the file. |
Line 31 saves the data to Truxton. | Line 31 saves the data to Truxton. | ||
| − | It will create a record in the <code>[[Entity Table | Entity]]</code> table in the database. | + | It will create a record in the <code><nowiki>[</nowiki>[[Entity Table|Entity]]<nowiki>]</nowiki></code> table in the database. |
Saving the artifact will cause Truxton to route it to any ETL that has subscribed to <code>[[Type_Artifact]]</code> messages. | Saving the artifact will cause Truxton to route it to any ETL that has subscribed to <code>[[Type_Artifact]]</code> messages. | ||
| + | |||
| + | Line 34 pauses your ETL until a new message arrives on it queue | ||
=Development and Debugging= | =Development and Debugging= | ||
| Line 83: | Line 85: | ||
# Now every time you run your program you will immediately receive that file to play with. | # Now every time you run your program you will immediately receive that file to play with. | ||
| − | < | + | <source lang="python" highlight="12"> |
import truxton | import truxton | ||
| Line 118: | Line 120: | ||
artifact.save() | artifact.save() | ||
| + | # Pause here until we get another message from the "wiley" message queue | ||
message = etl.getmessage() | message = etl.getmessage() | ||
if __name__ == "__main__": | if __name__ == "__main__": | ||
main() | main() | ||
| − | </ | + | </source> |
Revision as of 17:41, 24 November 2020
This sample shows the steps needed to implement a file exploitation ETL in Truxton.
Sample File Format
This sample will exploit a fake file format we call Acme. Acme Corporation is a known supplier of nefarious devices and explosives. Their file format begins with a five byte magic value followed by eleven bytes in a data structure. If the sixth byte in the file is 0x11 then it is a serial number file that uniquely identifies the user.
0000h: 88 77 66 55 00 11 22 33 44 55 66 77 88 99 AA BB 0010h: CC
Source Code
1 import truxton
2
3 def main():
4
5 etl = truxton.etl()
6 etl.name = "Acme Exploitation"
7 etl.description = "This exploits Acme Corporation data files"
8 etl.queue = "wiley"
9 etl.stage = 40
10 etl.addtype(11000)
11
12 message = etl.getmessage()
13
14 while message is not None:
15 file_in_truxton = message.file()
16
17 file_in_truxton.seek(5)
18
19 next_byte = file_in_truxton.read(1)
20 if next_byte[0] == 0x11:
21 # Serial Number. The next 8 bytes are a serial number
22 file_in_truxton.seek(6)
23 serial_number = file_in_truxton.read(8)
24
25 artifact = file_in_truxton.newartifact()
26 artifact.type = truxton.ENTITY_TYPE_SERIAL_NUMBER
27 artifact.value = serial_number.hex()
28 artifact.datatype = truxton.DATA_TYPE_uint8_t
29 artifact.offset = 6
30 artifact.length = 8
31 artifact.save()
32
33 # Pause here until we get another message from the "wiley" message queue
34 message = etl.getmessage()
35
36 if __name__ == "__main__":
37 main()
Code Walkthrough
Lines 5-10 setup the ETL. The message queue name will be "acme", we are an early stage and want to receive Acme files (11000 was chosen as the identifier of Acme files).
Line 12 starts the ETL logic and waits until a message arrives on the "wiley" queue.
Line 15 opens the file so we can read from it.
Lines 19-20 read the fifth byte in the file and checks it for validity.
Lines 25-31 creates an artifact (which will be stored in the [Entity] table in the database) and saves it to Truxton.
Line 26 sets the type of artifact to a serial number. This allows analysts to quickly find items of interest by their type.
Line 28 stores the format of how the serial number was stored in the file.
Line 31 saves the data to Truxton.
It will create a record in the [Entity] table in the database.
Saving the artifact will cause Truxton to route it to any ETL that has subscribed to Type_Artifact messages.
Line 34 pauses your ETL until a new message arrives on it queue
Development and Debugging
Truxton ETLs assume they are part of a processing stream instead of processing a file from the local system. This can slow your development cycle down. Here's one strategy that will make your development iterations quicker:
- Perform a load with your sample file in it
- Find the file using the desktop GUI, copy the file identifier.
- Stop the Truxton service
- Modify your program to use the
sendmefileid()function. - Now every time you run your program you will immediately receive that file to play with.
import truxton
def main():
etl = truxton.etl()
etl.name = "Acme Exploitation"
etl.description = "This exploits Acme Corporation data files"
etl.queue = "wiley"
etl.stage = 40
etl.addtype(11000)
etl.sendmefileid("5f06ef4d-03dd-5258-2758-378e00000011")
message = etl.getmessage()
while message is not None:
file_in_truxton = message.file()
file_in_truxton.seek(5)
next_byte = file_in_truxton.read(1)
if next_byte[0] == 0x11:
# Serial Number. The next 8 bytes are a serial number
file_in_truxton.seek(6)
serial_number = file_in_truxton.read(8)
artifact = file_in_truxton.newartifact()
artifact.type = truxton.ENTITY_TYPE_SERIAL_NUMBER
artifact.value = serial_number.hex()
artifact.datatype = truxton.DATA_TYPE_uint8_t
artifact.offset = 6
artifact.length = 8
artifact.save()
# Pause here until we get another message from the "wiley" message queue
message = etl.getmessage()
if __name__ == "__main__":
main()
