Difference between revisions of "Loading Strategies"
| Line 4: | Line 4: | ||
You can even script the load. | You can even script the load. | ||
| − | = | + | =Single Media= |
When you have just a few pieces of media to load, you can use the Desktop Gui or Windows Explorer (right-button Load...). | When you have just a few pieces of media to load, you can use the Desktop Gui or Windows Explorer (right-button Load...). | ||
Manually starting a load is for single-user environments where everything is on a single machine. | Manually starting a load is for single-user environments where everything is on a single machine. | ||
| Line 14: | Line 14: | ||
</source> | </source> | ||
| − | = | + | =Several Media= |
When you get to the point where you have more than several hard drives to process at a time, a dedicated loader machine is used. | When you get to the point where you have more than several hard drives to process at a time, a dedicated loader machine is used. | ||
At this level, overloading the machine becomes a problem. | At this level, overloading the machine becomes a problem. | ||
| Line 21: | Line 21: | ||
This allows you to balance the resources of your machine with the amount of data to load. | This allows you to balance the resources of your machine with the amount of data to load. | ||
| − | = | + | =Hundreds of Media= |
| − | When you have a never ending stream of incoming media, you need something even more automatic than [[Load List | load lists]]. | + | When you have a never ending stream of incoming media, you need something even more automatic than [[Load List|load lists]]. |
Multiple machines dedicated to loading are pressed into service. | Multiple machines dedicated to loading are pressed into service. | ||
Like other ETL processes, the loader has its own message queue that loading tasks can be sent to. | Like other ETL processes, the loader has its own message queue that loading tasks can be sent to. | ||
When a load process on one of those machines finishes, it will grab another load task from the message queue. | When a load process on one of those machines finishes, it will grab another load task from the message queue. | ||
| + | One way of dealing with the volume is to use a tool for automating workflows such as [https://en.wikipedia.org/wiki/Apache_NiFi NiFi]. | ||
| + | |||
| + | =Quality Controlled= | ||
| + | In this strategy, you have broken the exploitation process into separate loading and publishing phases. | ||
| + | There is a separate server that the investigators use to analyze the Truxton output. | ||
| + | Someone will | ||
Revision as of 08:10, 24 January 2021
The way you load data into Truxton mostly depends on the amount of data you have to load. If you're one of those kinds of people that likes to type, you can use the command line. From Windows File Explorer, you can right button on a file or folder and load it into Truxton. You can even script the load.
Single Media
When you have just a few pieces of media to load, you can use the Desktop Gui or Windows Explorer (right-button Load...). Manually starting a load is for single-user environments where everything is on a single machine. The amount of data is a couple of hard drives, half a dozen phones, etc.
You can also load from the command line.
"C:\Program Files\Truxton\Loader\Load.exe" WinXP.E01
Several Media
When you get to the point where you have more than several hard drives to process at a time, a dedicated loader machine is used. At this level, overloading the machine becomes a problem. There's too much data to load and not enough time to sit and watch the loaders. Using a Load List will allow you to specify the media to load and how many simultaneous load processes to use. This allows you to balance the resources of your machine with the amount of data to load.
Hundreds of Media
When you have a never ending stream of incoming media, you need something even more automatic than load lists. Multiple machines dedicated to loading are pressed into service. Like other ETL processes, the loader has its own message queue that loading tasks can be sent to. When a load process on one of those machines finishes, it will grab another load task from the message queue. One way of dealing with the volume is to use a tool for automating workflows such as NiFi.
Quality Controlled
In this strategy, you have broken the exploitation process into separate loading and publishing phases. There is a separate server that the investigators use to analyze the Truxton output. Someone will