Difference between revisions of "Load List"

From truxwiki.com
Jump to navigation Jump to search
 
(23 intermediate revisions by the same user not shown)
Line 1: Line 1:
−
A load list is a simple text file that allows you to script the load process. These are usually used when you have many pieces of media to load on a single machine. You can list the path to the media to load and Truxton will load as many of them as possible at a time until they are all loaded. You don’t have to hang around and start the next piece of media when one finishes.
+
A load list is a simple text file that allows you to script the load process.
 +
These are usually used when you have many pieces of media to load on a single machine.
 +
You can list the path to the media to load and Truxton will load as many of them as possible at a time until they are all loaded.
 +
You don’t have to hang around and start the next piece of media when one finishes.
  
 
It is not user friendly in that it is not self-explanatory. It was designed to be generated by scripts. The format of the file is a single letter record identifier followed by a colon followed by the parameters for that record type.
 
It is not user friendly in that it is not self-explanatory. It was designed to be generated by scripts. The format of the file is a single letter record identifier followed by a colon followed by the parameters for that record type.
Line 5: Line 8:
 
=Fields=
 
=Fields=
 
==Investigation==
 
==Investigation==
−
The investigation line specifies the investigation that the following media should belong to. You may specify a globally unique identifier if you wish otherwise Truxton will generate one for you. If you want the media in this load list file to belong to an existing investigation, simply put the identifier of that investigation on this line. Once an investigation has been set, all media specified in the load list will belong to that investigation until another investigation line is present.
+
The investigation line specifies the investigation that the following media should belong to.
 +
You may specify a [https://en.wikipedia.org/wiki/Universally_unique_identifier globally unique identifier] if you wish, otherwise Truxton will generate one for you.
 +
If you want the media in this load list file to belong to an existing investigation, put the identifier of that investigation on this line.
 +
 
 +
Once an investigation has been set, all media specified in the load list will belong to that investigation until another investigation line is present.
  
 
===Samples===
 
===Samples===
−
<syntaxhighlight lang="text">
+
<source lang="text">
 
I: This is My Case
 
I: This is My Case
−
</syntaxhighlight>
+
</source>
 
A new investigation will be created with the name "This is My Case" with a random identifier.
 
A new investigation will be created with the name "This is My Case" with a random identifier.
 
<hr />
 
<hr />
  
−
<syntaxhighlight lang="text">
+
<source lang="text">
 
I: Case 14 2CBA17E4-038A-4828-8A0D-D221F943B3F9
 
I: Case 14 2CBA17E4-038A-4828-8A0D-D221F943B3F9
−
</syntaxhighlight>
+
</source>
 
A new investigation will be created with the name of "Case 14" and an identifier of "2CBA17E4-038A-4828-8A0D-D221F943B3F9"
 
A new investigation will be created with the name of "Case 14" and an identifier of "2CBA17E4-038A-4828-8A0D-D221F943B3F9"
 
<hr />
 
<hr />
  
−
<syntaxhighlight lang="text">
+
<source lang="text">
 
i: {2CBA17E4-038A-4828-8A0D-D221F943B3F9}
 
i: {2CBA17E4-038A-4828-8A0D-D221F943B3F9}
−
</syntaxhighlight>
+
</source>
 
The media specified in this load list file will be made part of an existing investigation with the id of "2CBA17E4-038A-4828-8A0D-D221F943B3F9."
 
The media specified in this load list file will be made part of an existing investigation with the id of "2CBA17E4-038A-4828-8A0D-D221F943B3F9."
  
 
==Number of Loaders==
 
==Number of Loaders==
−
The loaders line controls the number of load processes that will be used to load the media specified in the load list. This setting will remain in effect until the next loaders line or the end of file has been reached.
+
The loaders line controls the number of load processes that will be used to load the media specified in the load list.
 +
This setting will remain in effect until the next loaders line or the end of file has been reached.
 +
Number of loaders is ignored when the enqueue (<code>[[#Enqueue |Q:]]</code>) setting is specified.
 +
When using a load queue, the total number of loaders on your network is controlled by the <code>[[TruxtonService.xml#elt|instances]]</code> attribute of the <code>etl</code> element for the <code>load</code> process in the <code>[[TruxtonService.xml]]</code> file on each loader machine.
 +
The sum of the <code>instances</code> values from the different loader machines will be the total number of loaders used to process the media.
 +
 
 
===Samples===
 
===Samples===
−
<syntaxhighlight lang="text">
+
<source lang="text">
 
L: 4
 
L: 4
−
</syntaxhighlight>
+
</source>
 
This will use four load processes to get the media in this list loaded.
 
This will use four load processes to get the media in this list loaded.
 
<hr />
 
<hr />
  
−
<syntaxhighlight lang="text">
+
<source lang="text">
 
L: 0
 
L: 0
−
</syntaxhighlight>
+
</source>
 
This will set the number of load processes equal to the number of CPUs in the loader machine.
 
This will set the number of load processes equal to the number of CPUs in the loader machine.
  
Line 42: Line 54:
 
You can specify a configuration file to use. This can be used in conjunction with the path to media option. They are not exclusive.
 
You can specify a configuration file to use. This can be used in conjunction with the path to media option. They are not exclusive.
 
===Samples===
 
===Samples===
−
<syntaxhighlight lang="text">
+
<source lang="text">
 
C: c:\Configs\Triage.xml
 
C: c:\Configs\Triage.xml
−
</syntaxhighlight>
+
</source>
−
This will cause the loaders to use configuration parameters as specified in the "c:\Configs\Triage.xml" file.
+
This will cause the loaders to use configuration parameters as specified in the <code>c:\Configs\Triage.xml</code> file.
  
−
==Hash List Directory==
+
==Hash Set Directory==
−
The hash list directory line sets the path to the folder that contains hash lists for eliminating file contents during the load.
+
The [[Hash Set | hash set]] directory line sets the path to the folder that contains hash sets for eliminating file contents during the load.
  
 
===Samples===
 
===Samples===
−
<syntaxhighlight lang="text">
+
<source lang="text">
 
H: c:\Hashes
 
H: c:\Hashes
−
</syntaxhighlight>
+
</source>
 
This will use all hash sets found in the "C:\Hashes" folder
 
This will use all hash sets found in the "C:\Hashes" folder
 
<hr />
 
<hr />
−
<syntaxhighlight lang="text">
+
<source lang="text">
 
H: \\hashserver\files\2016
 
H: \\hashserver\files\2016
−
</syntaxhighlight>
+
</source>
 
This will use all hash sets found in the shared folder "\\hashserver\files\2016"
 
This will use all hash sets found in the shared folder "\\hashserver\files\2016"
 
It is '''not''' recommended that you put hashsets on a server.
 
It is '''not''' recommended that you put hashsets on a server.
Line 66: Line 78:
 
This line contains the name and optional identifier for the media to be loaded. If no identifier is specified, one will be generated for you.
 
This line contains the name and optional identifier for the media to be loaded. If no identifier is specified, one will be generated for you.
 
===Samples===
 
===Samples===
−
<syntaxhighlight lang="text">
+
<source lang="text">
 
M: Laptop Drive
 
M: Laptop Drive
−
</syntaxhighlight>
+
</source>
 
This will set the name of the media to "Laptop Drive." The identifier for the media will be completely random.
 
This will set the name of the media to "Laptop Drive." The identifier for the media will be completely random.
 
<hr />
 
<hr />
−
<syntaxhighlight lang="text">
+
<source lang="text">
 
M: Bob's Phone 2CBA17E4-038A-4828-8A0D-D221F943B3F9
 
M: Bob's Phone 2CBA17E4-038A-4828-8A0D-D221F943B3F9
−
</syntaxhighlight>
+
</source>
 
This will set the name of the media to "Bob's Phone" and set the identifier to "2CBA17E4-038A-4828-8A0D-D221F943B3F9"
 
This will set the name of the media to "Bob's Phone" and set the identifier to "2CBA17E4-038A-4828-8A0D-D221F943B3F9"
  
Line 80: Line 92:
  
 
===Samples===
 
===Samples===
−
<syntaxhighlight lang="text">
+
<source lang="text">
 
G: -loadas 514
 
G: -loadas 514
−
</syntaxhighlight>
+
</source>
−
This will add the <code>loadas</code> command line argument when the load process is spawned.
+
This will add the <code>loadas</code> command line argument when the load process is spawned telling it to treat the media as a [[Type_Master_Boot_Record|Master Boot Record]].
  
 
==Execute Before==
 
==Execute Before==
−
This allows you to specify a [https://docs.microsoft.com/en-us/powershell/ PowerShell] script or batch file to execute before the load process is spawned. Once the script completes, the loader is free to execute.
+
This allows you to specify a [https://docs.microsoft.com/en-us/powershell/ PowerShell] script, [https://www.python.org/ Python] script, or batch file to execute before the load process is spawned.
 +
Once the script completes, the loader is free to execute.
 +
You can run multiple scripts by adding one script per line.
  
 
===Samples===
 
===Samples===
−
<syntaxhighlight lang="text">
+
<source lang="text">
 
A: BeforeBobLoads.ps1
 
A: BeforeBobLoads.ps1
−
</syntaxhighlight>
+
</source>
 
This script will execute before the loader process is spawned.
 
This script will execute before the loader process is spawned.
 +
 +
<source lang="text">
 +
A: StartBillingRecord.py customer=THX1138 category=ForensicServices
 +
A: ClearTemporaryFolders.cmd
 +
A: SendStartMessage.bat
 +
A: BeforeBobLoads.ps1
 +
</source>
 +
These scripts will execute sequentially before the loader process is spawned.
  
 
==Execute After==
 
==Execute After==
−
This allows you to specify a [https://docs.microsoft.com/en-us/powershell/ PowerShell] script or batch file to execute after the load process exits.
+
This allows you to specify a [https://docs.microsoft.com/en-us/powershell/ PowerShell] script, [https://www.python.org/ Python] script, or batch file to execute after the [[Load|load]] process exits.
 +
You can run multiple scripts by adding one script per line.
 +
 
 +
Exploitation of the media is not yet complete when these scripts execute.
 +
 
 
===Samples===
 
===Samples===
−
<syntaxhighlight lang="text">
+
<source lang="text">
 
Z: AfterBobLoads.cmd
 
Z: AfterBobLoads.cmd
−
</syntaxhighlight>
+
</source>
 
This script will execute once the load completes.
 
This script will execute once the load completes.
 +
 +
<source lang="text">
 +
Z: CompleteBillingRecord.py customer=THX1138
 +
Z: ClearTemporaryFolders.cmd
 +
Z: SendEndMessage.bat
 +
Z: AfterBobLoads.ps1
 +
</source>
 +
These scripts will execute sequentially before the loader process is spawned.
  
 
==Path==
 
==Path==
 
This is the line that causes the load process to be spawned. This holds the path to the media to be loaded.
 
This is the line that causes the load process to be spawned. This holds the path to the media to be loaded.
 
===Samples===
 
===Samples===
−
<syntaxhighlight lang="text">
+
<source lang="text">
 
P: c:\Images\WinXP.E01
 
P: c:\Images\WinXP.E01
−
</syntaxhighlight>
+
</source>
 
This will load the "WinXP.E01" file.
 
This will load the "WinXP.E01" file.
 
<hr />
 
<hr />
  
−
<syntaxhighlight lang="text">
+
<source lang="text">
 
P: c:\Data\Files 1
 
P: c:\Data\Files 1
−
</syntaxhighlight>
+
</source>
 
This will load the folder "Files 1"
 
This will load the folder "Files 1"
 +
 +
==Enqueue==
 +
If present, it signifies that you would rather have this entry placed on the loader queue so any loader on the network can load it.
 +
This is a global setting.
 +
If specified, ALL entries in the load list will be placed onto the loader queue.
 +
This will cause the <code>[[#Number of Loaders |L:]]</code> to be ignored.
 +
When using a load queue, the total number of loaders on your network is controlled by the <code>[[TruxtonService.xml#elt|instances]]</code> attribute of the <code>etl</code> element for the <code>load</code> process in the <code>[[TruxtonService.xml]]</code> file on each loader machine.
 +
The sum of the <code>instances</code> values from the different loader machines will be the total number of loaders used to process the media.
 +
 +
===Samples===
 +
The following tells Truxton to allow any loader machine on the network to load the incoming data.
 +
 +
<source lang="text">
 +
Q: load
 +
P: \\Server\Incoming Data\East Coast\Virginia\Case 1234
 +
</source>
  
 
==Complete Sample Load List==
 
==Complete Sample Load List==
 
The following load list will use one load process to exhaust the load list. Two pieces of media will be loaded as part of the [https://comicvine.gamespot.com/hong-kong-cavaliers/4060-57943/ "Hong Kong Cavaliers"] investigation. A PowerShell script will be executed before [https://www.imdb.com/title/tt0086856/characters/nm0000289 Penny Priddy's] phone is loaded. [https://www.imdb.com/title/tt0086856/characters/nm0001475 Emilio Lizardo's] hard drive will be loaded after Penny’s phone load completes. It will be loaded as a Master Boot Record ([[Type_Master_Boot_Record|file type 514]]) and [[Media Types|media type]] of Hard Drive (3).
 
The following load list will use one load process to exhaust the load list. Two pieces of media will be loaded as part of the [https://comicvine.gamespot.com/hong-kong-cavaliers/4060-57943/ "Hong Kong Cavaliers"] investigation. A PowerShell script will be executed before [https://www.imdb.com/title/tt0086856/characters/nm0000289 Penny Priddy's] phone is loaded. [https://www.imdb.com/title/tt0086856/characters/nm0001475 Emilio Lizardo's] hard drive will be loaded after Penny’s phone load completes. It will be loaded as a Master Boot Record ([[Type_Master_Boot_Record|file type 514]]) and [[Media Types|media type]] of Hard Drive (3).
  
−
<syntaxhighlight lang="text">
+
<source lang="text">
 
L: 1
 
L: 1
 
I: Hong Kong Cavaliers
 
I: Hong Kong Cavaliers
Line 130: Line 180:
 
G: -loadas 514 -mt 3
 
G: -loadas 514 -mt 3
 
P: \\ImageServer\Hard Drives\EML.dd
 
P: \\ImageServer\Hard Drives\EML.dd
 +
</source>
 +
 +
=Creating A Load List=
 +
All you need to create a load list is a text editor.
 +
Here are some ideas on creating a list.
 +
 +
==Folder Structure of Disk Images==
 +
In this scenario, we have a deep folder structure with E01 files in them.
 +
The most basic way to create the load list is to get a directory listing into a file then edit that file.
 +
 +
<source lang="bat">
 +
cd C:\Storage\Incoming
 +
dir /s /b *.E01 >loadlist.txt
 +
</source>
 +
 +
This will result in a text file that contains the full path to every E01 in the folders.
 +
 +
<pre>C:\Storage\Incoming\HDD1\Case 1\Hitachi\DiskImage.E01
 +
C:\Storage\Incoming\HDD1\Case 1\Western Digital\DiskImage.E01
 +
C:\Storage\Incoming\HDD2\Omega\Seagate\DiskImage.E01</pre>
 +
 +
Edit the <code>loadlist.txt</code> file in an editor.
 +
Search and replace the <code>C:\</code> text with <code>P: C:\</code>
 +
 +
<pre>P: C:\Storage\Incoming\HDD1\Case 1\Hitachi\DiskImage.E01
 +
P: C:\Storage\Incoming\HDD1\Case 1\Western Digital\DiskImage.E01
 +
P: C:\Storage\Incoming\HDD2\Omega\Seagate\DiskImage.E01</pre>
 +
 +
You now have a functional load list.
  
−
</syntaxhighlight>
+
=Finding Python=
 +
If you specify a Python script file to execute before or after a load, the loader must find the Python executable.
 +
It does this by first retrieving the <code>python</code> [[Configuration System|setting]].
 +
If that doesn't work, it will then read the <code>HKEY_CLASSES_ROOT\Python.File\shell\open\command</code> from the [https://docs.microsoft.com/en-us/windows/win32/sysinfo/registry Registry].

Latest revision as of 06:44, 5 May 2023

A load list is a simple text file that allows you to script the load process. These are usually used when you have many pieces of media to load on a single machine. You can list the path to the media to load and Truxton will load as many of them as possible at a time until they are all loaded. You don’t have to hang around and start the next piece of media when one finishes.

It is not user friendly in that it is not self-explanatory. It was designed to be generated by scripts. The format of the file is a single letter record identifier followed by a colon followed by the parameters for that record type.

Fields

Investigation

The investigation line specifies the investigation that the following media should belong to. You may specify a globally unique identifier if you wish, otherwise Truxton will generate one for you. If you want the media in this load list file to belong to an existing investigation, put the identifier of that investigation on this line.

Once an investigation has been set, all media specified in the load list will belong to that investigation until another investigation line is present.

Samples

I: This is My Case

A new investigation will be created with the name "This is My Case" with a random identifier. 

I: Case 14 2CBA17E4-038A-4828-8A0D-D221F943B3F9

A new investigation will be created with the name of "Case 14" and an identifier of "2CBA17E4-038A-4828-8A0D-D221F943B3F9" 

i: {2CBA17E4-038A-4828-8A0D-D221F943B3F9}

The media specified in this load list file will be made part of an existing investigation with the id of "2CBA17E4-038A-4828-8A0D-D221F943B3F9."

Number of Loaders

The loaders line controls the number of load processes that will be used to load the media specified in the load list. This setting will remain in effect until the next loaders line or the end of file has been reached. Number of loaders is ignored when the enqueue (Q:) setting is specified. When using a load queue, the total number of loaders on your network is controlled by the instances attribute of the etl element for the load process in the TruxtonService.xml file on each loader machine. The sum of the instances values from the different loader machines will be the total number of loaders used to process the media.

Samples

L: 4

This will use four load processes to get the media in this list loaded. 

L: 0

This will set the number of load processes equal to the number of CPUs in the loader machine.

Configuration

You can specify a configuration file to use. This can be used in conjunction with the path to media option. They are not exclusive.

Samples

C: c:\Configs\Triage.xml

This will cause the loaders to use configuration parameters as specified in the c:\Configs\Triage.xml file.

Hash Set Directory

The hash set directory line sets the path to the folder that contains hash sets for eliminating file contents during the load.

Samples

H: c:\Hashes

This will use all hash sets found in the "C:\Hashes" folder 

H: \\hashserver\files\2016

This will use all hash sets found in the shared folder "\\hashserver\files\2016" It is not recommended that you put hashsets on a server. During a load, these files are accessed frequently and should be put on the fastest storage you have on the loader machine.

Media

This line contains the name and optional identifier for the media to be loaded. If no identifier is specified, one will be generated for you.

Samples

M: Laptop Drive

This will set the name of the media to "Laptop Drive." The identifier for the media will be completely random. 

M: Bob's Phone 2CBA17E4-038A-4828-8A0D-D221F943B3F9

This will set the name of the media to "Bob's Phone" and set the identifier to "2CBA17E4-038A-4828-8A0D-D221F943B3F9"

Command Line Arguments

This allows you to set any command line argument for use with this media.

Samples

G: -loadas 514

This will add the loadas command line argument when the load process is spawned telling it to treat the media as a Master Boot Record.

Execute Before

This allows you to specify a PowerShell script, Python script, or batch file to execute before the load process is spawned. Once the script completes, the loader is free to execute. You can run multiple scripts by adding one script per line.

Samples

A: BeforeBobLoads.ps1

This script will execute before the loader process is spawned. 

A: StartBillingRecord.py customer=THX1138 category=ForensicServices
A: ClearTemporaryFolders.cmd
A: SendStartMessage.bat
A: BeforeBobLoads.ps1

These scripts will execute sequentially before the loader process is spawned.

Execute After

This allows you to specify a PowerShell script, Python script, or batch file to execute after the load process exits. You can run multiple scripts by adding one script per line.

Exploitation of the media is not yet complete when these scripts execute.

Samples

Z: AfterBobLoads.cmd

This script will execute once the load completes. 

Z: CompleteBillingRecord.py customer=THX1138
Z: ClearTemporaryFolders.cmd
Z: SendEndMessage.bat
Z: AfterBobLoads.ps1

These scripts will execute sequentially before the loader process is spawned.

Path

This is the line that causes the load process to be spawned. This holds the path to the media to be loaded.

Samples

P: c:\Images\WinXP.E01

This will load the "WinXP.E01" file. 

P: c:\Data\Files 1

This will load the folder "Files 1"

Enqueue

If present, it signifies that you would rather have this entry placed on the loader queue so any loader on the network can load it. This is a global setting. If specified, ALL entries in the load list will be placed onto the loader queue. This will cause the L: to be ignored. When using a load queue, the total number of loaders on your network is controlled by the instances attribute of the etl element for the load process in the TruxtonService.xml file on each loader machine. The sum of the instances values from the different loader machines will be the total number of loaders used to process the media.

Samples

The following tells Truxton to allow any loader machine on the network to load the incoming data. 

Q: load
P: \\Server\Incoming Data\East Coast\Virginia\Case 1234

Complete Sample Load List

The following load list will use one load process to exhaust the load list. Two pieces of media will be loaded as part of the "Hong Kong Cavaliers" investigation. A PowerShell script will be executed before Penny Priddy's phone is loaded. Emilio Lizardo's hard drive will be loaded after Penny’s phone load completes. It will be loaded as a Master Boot Record (file type 514) and media type of Hard Drive (3). 

L: 1
I: Hong Kong Cavaliers

A: AddPhone.ps1
M: Penny Priddy’s Phone
P: \\ImageServer\Phones\PP.E01

M: Emilio Lizardo’s Hard Drive
G: -loadas 514 -mt 3
P: \\ImageServer\Hard Drives\EML.dd

Creating A Load List

All you need to create a load list is a text editor. Here are some ideas on creating a list.

Folder Structure of Disk Images

In this scenario, we have a deep folder structure with E01 files in them. The most basic way to create the load list is to get a directory listing into a file then edit that file. 

cd C:\Storage\Incoming
dir /s /b *.E01 >loadlist.txt

This will result in a text file that contains the full path to every E01 in the folders. 

C:\Storage\Incoming\HDD1\Case 1\Hitachi\DiskImage.E01
C:\Storage\Incoming\HDD1\Case 1\Western Digital\DiskImage.E01
C:\Storage\Incoming\HDD2\Omega\Seagate\DiskImage.E01

Edit the loadlist.txt file in an editor. Search and replace the C:\ text with P: C:\ 

P: C:\Storage\Incoming\HDD1\Case 1\Hitachi\DiskImage.E01
P: C:\Storage\Incoming\HDD1\Case 1\Western Digital\DiskImage.E01
P: C:\Storage\Incoming\HDD2\Omega\Seagate\DiskImage.E01

You now have a functional load list.

Finding Python

If you specify a Python script file to execute before or after a load, the loader must find the Python executable. It does this by first retrieving the python setting. If that doesn't work, it will then read the HKEY_CLASSES_ROOT\Python.File\shell\open\command from the Registry.

Retrieved from "https://truxwiki.com/index.php?title=Load_List&oldid=6293"