Difference between revisions of "Configuration System"

From truxwiki.com
Jump to navigation Jump to search
 
(12 intermediate revisions by the same user not shown)
Line 10: Line 10:
 
The following are the nine different places that a parameter can come from. The lower the number, the greater the precedence that option has. The harder it is for a user to define an option, the greater weight that option will be given. It is harder for a user to specify an option on a command line because they must type it out. The user exerts no effort whatsoever to use a default value compiled into programs. Therefore, the command line option will be used instead of the compiled default.  
 
The following are the nine different places that a parameter can come from. The lower the number, the greater the precedence that option has. The harder it is for a user to define an option, the greater weight that option will be given. It is harder for a user to specify an option on a command line because they must type it out. The user exerts no effort whatsoever to use a default value compiled into programs. Therefore, the command line option will be used instead of the compiled default.  
  
−
# Command Line Parameters
+
# Command Line Parameters - Highest priority
 
# Command Line Configuration File
 
# Command Line Configuration File
 
# Command Line Database
 
# Command Line Database
Line 18: Line 18:
 
# Database by Machine
 
# Database by Machine
 
# Truxton Settings File
 
# Truxton Settings File
−
# Compiled Defaults
+
# Compiled Defaults - Lowest priority
 +
 
 +
==Why So Many Choices?==
 +
There are many ways to run processes.
 +
So many scenarios require different ways to configure.
 +
If you host an [[ETL Stages#ETLs and Their Stages|ETL process]] in a Docker image, it is far easier to insert environment variables into the deployment instead of pushing configuration files into the image.
 +
This allows you to create configuration-free Docker images containing the ETL processes you want.
 +
Writing a one-time script to process something in Truxton can use command line arguments for customization instead of editing configuration files.
 +
 
 +
Truxton tries to make it possible to configure the processes however you my need to run them.
  
 
==Command Line Parameters==
 
==Command Line Parameters==
Line 52: Line 61:
 
You can put the name of a configuration file on the command line.
 
You can put the name of a configuration file on the command line.
  
−
<syntaxhighlight lang="bat">
+
<source lang="bat">
 
Identify.exe my_settings.xml
 
Identify.exe my_settings.xml
−
</syntaxhighlight>
+
</source>
  
 
The <code>my_settings.xml</code> would be assumed to be a file name since it was not part of an option.
 
The <code>my_settings.xml</code> would be assumed to be a file name since it was not part of an option.
−
That file would then be opened, parsed as [https://en.wikipedia.org/wiki/XML XML] and anything found in a <code>truxton_options</code> section would be used as configuration options.
+
That file would then be opened, parsed as [https://en.wikipedia.org/wiki/XML XML] and anything found in a <code>truxton_options</code> section will be used as configuration options.
 
All XML element names used in Truxton are lower case.
 
All XML element names used in Truxton are lower case.
  
 
The settings would look like this:
 
The settings would look like this:
−
<syntaxhighlight lang="xml">
+
<source lang="xml">
 
<truxton_options>
 
<truxton_options>
 
   <w>1</w>
 
   <w>1</w>
Line 67: Line 76:
 
   <z>3 4 and 5</z>
 
   <z>3 4 and 5</z>
 
</truxton_options>
 
</truxton_options>
−
</syntaxhighlight>
+
</source>
  
 
==Command Line Database==
 
==Command Line Database==
Line 90: Line 99:
 
These variables must have names that begin with <code>Truxton_</code>
 
These variables must have names that begin with <code>Truxton_</code>
 
For example:
 
For example:
−
<syntaxhighlight lang="bat">
+
<source lang="bat">
 
set Truxton_w=1
 
set Truxton_w=1
 
set Truxton_xx=abc
 
set Truxton_xx=abc
 
set Truxton_z="3 4 and 5"
 
set Truxton_z="3 4 and 5"
 
Load.exe disk.dat
 
Load.exe disk.dat
−
</syntaxhighlight>
+
</source>
  
 
==ETL Configuration File==
 
==ETL Configuration File==
Line 113: Line 122:
 
==Database by Machine==
 
==Database by Machine==
 
Each installation of Truxton has a globally unique machine identifier.
 
Each installation of Truxton has a globally unique machine identifier.
−
This can be found in the <code>machineid</code> element of the <code>TruxtonSettings.XML</code> file.
+
This can be found in the <code>[[TruxtonObject#machineid:_str]]</code> element of the <code>TruxtonSettings.XML</code> file.
−
Truxton will connect to the database and query the <code>Setting</code> table for a record with the machine identifier in the <code>ID</code> column.
+
Truxton will connect to the database and query the <code>[Setting]</code> table for a record with the machine identifier in the <code>[ID]</code> column.
 
If found, the XML in the <code>Value</code> column will be parsed for options.
 
If found, the XML in the <code>Value</code> column will be parsed for options.
  
Line 134: Line 143:
 
With nine different places to get options from, how could anything go wrong? If you need help debugging why an option isn't getting the value you want, use the <code>debugoptions</code> option.
 
With nine different places to get options from, how could anything go wrong? If you need help debugging why an option isn't getting the value you want, use the <code>debugoptions</code> option.
  
−
<syntaxhighlight lang="bat">
+
<source lang="bat">
 
Identify.exe -debugoptions 1
 
Identify.exe -debugoptions 1
−
</syntaxhighlight>
+
</source>
  
 
This will cause an installation log to be created and the options written to it.
 
This will cause an installation log to be created and the options written to it.
Line 143: Line 152:
 
==Sample Options Log==
 
==Sample Options Log==
 
<pre>
 
<pre>
−
Log file created by "C:\Program Files\Truxton\Loader\Identify.exe"
+
Log file created 2022-07-25 12:26:07 by "C:\Program Files\Truxton\Loader\Identify.exe" running as Sam on the machine named DESKTOP-5IFOKKS (192.168.1.88).
−
2019-10-16T12:31:21Z 4454-6528 : Processing "C:\ProgramData\Truxton\Settings\TruxtonSettings.xml" as TruxtonSettings.xml(C:\ProgramData\Truxton\Settings\TruxtonSettings.xml)
+
2022-07-25T16:26:06Z 2CAC-84D4 : Parsing 3 command line arguments
−
2019-10-16T12:31:21Z 4454-6528 : Getting Options from Truxton Database
+
2022-07-25T16:26:06Z 2CAC-84D4 : First command line argument was something that ended in .exe, skipping it
−
2019-10-16T12:31:21Z 4454-6528 : Parsing 3 command line arguments
+
2022-07-25T16:26:06Z 2CAC-84D4 : Beginning to parse the command line arguments at array index 1
−
2019-10-16T12:31:21Z 4454-6528 : Beginning to parse the command line arguments at array index 0
+
2022-07-25T16:26:06Z 2CAC-84D4 : command_line_argument[ 1 ] = "-debugoptions"
−
2019-10-16T12:31:21Z 4454-6528 : command_line_argument[ 0 ] = "identify"
+
2022-07-25T16:26:06Z 2CAC-84D4 : command_line_argument[ 2 ] = "1" is the value of option "debugoptions"
−
2019-10-16T12:31:21Z 4454-6528 : command_line_argument[ 0 ] ("identify") is being added as a NotOption.
+
2022-07-25T16:26:06Z 2CAC-84D4 : No existing option named "debugoptions" was found, creating a new one
−
2019-10-16T12:31:21Z 4454-6528 : command_line_argument[ 0 ] ("identify") was added as a NotOption number 0.
+
2022-07-25T16:26:06Z 2CAC-84D4 : Finished parsing 3 command line arguments, there are now 0 NotOptions
−
2019-10-16T12:31:21Z 4454-6528 : command_line_argument[ 1 ] = "-debugoptions"
+
2022-07-25T16:26:06Z 2CAC-84D4 : Processing "C:\ProgramData\Truxton\Settings\TruxtonSettings.xml" as TruxtonSettings.xml(C:\ProgramData\Truxton\Settings\TruxtonSettings.xml)
−
2019-10-16T12:31:21Z 4454-6528 : command_line_argument[ 2 ] = "1" is the value of option "debugoptions"
+
2022-07-25T16:26:06Z 2CAC-84D4 : Processing "C:\Program Files\Truxton\Loader\Identify.config" as the ETL.config file
−
2019-10-16T12:31:21Z 4454-6528 : No existing option named "debugoptions" was found, creating a new one
+
2022-07-25T16:26:06Z 2CAC-84D4 : Can't parse "C:\Program Files\Truxton\Loader\Identify.config" as XML
−
2019-10-16T12:31:21Z 4454-6528 : Finished parsing 3 command line arguments, there are now 1 NotOptions
+
2022-07-25T16:26:06Z 2CAC-84D4 : Processing "C:\ProgramData\Truxton\Settings\TruxtonSettings.machine" as the ETL.machine file
−
2019-10-16T12:31:21Z 4454-6528 : Processing "C:\Program Files\Truxton\Loader\Identify.config" as the ETL.config file
+
2022-07-25T16:26:06Z 2CAC-84D4 : Can't parse "C:\ProgramData\Truxton\Settings\TruxtonSettings.machine" as XML
−
2019-10-16T12:31:21Z 4454-6528 : Can't parse "C:\Program Files\Truxton\Loader\Identify.config" as XML
+
2022-07-25T16:26:06Z 2CAC-84D4 : Getting Options from Truxton Database
−
2019-10-16T12:31:21Z 4454-6528 : Processing "C:\ProgramData\Truxton\Settings\TruxtonSettings.machine" as the ETL.machine file
+
2022-07-25T16:26:06Z 2CAC-84D4 : All Option Dump (The first entry of duplicate lines is the value that will be used by the program.)
−
2019-10-16T12:31:21Z 4454-6528 : Can't parse "C:\ProgramData\Truxton\Settings\TruxtonSettings.machine" as XML
+
Process "C:\Program Files\Truxton\Loader\Identify.exe"
−
2019-10-16T12:31:21Z 4454-6528 : All Option Dump (The first entry of duplicate lines is the value that will be used by the program.)
+
createthedatabase : "1" from TruxtonSettings.xml XML element name "truxton_options.CreateTheDatabase" beginning at line 6 column 3 byte 382 of "C:\ProgramData\Truxton\Settings\TruxtonSettings.xml"
 +
datadir : "C:\Truxton Data" from TruxtonSettings.xml XML element name "truxton_options.datadir" beginning at line 4 column 3 byte 281 of "C:\ProgramData\Truxton\Settings\TruxtonSettings.xml"
 +
dbconnectionstring : "Host=localhost;Port=5432;Database=Truxton;Username=postgres;Password=Truxton4n6;" from TruxtonSettings.xml XML element name "truxton_options.dbconnectionstring" beginning at line 2 column 3 byte 21 of "C:\ProgramData\Truxton\Settings\TruxtonSettings.xml"
 +
debugoptions : "1" from Command Line argument 2 (zero is the first)
 +
filegroupshavebeeninitialized : "1" from TruxtonSettings.xml XML element name "truxton_options.FileGroupsHaveBeenInitialized" beginning at line 16 column 3 byte 833 of "C:\ProgramData\Truxton\Settings\TruxtonSettings.xml"
 +
machineid : "671B7D4B-0D2E-4A00-84D6-FD9F6CA1DEBA" from TruxtonSettings.xml XML element name "truxton_options.machineid" beginning at line 5 column 3 byte 319 of "C:\ProgramData\Truxton\Settings\TruxtonSettings.xml"
 +
mbconnectionstring : "Host=localhost;Port=5432;Database=TruxtonMessageBus;Username=postgres;Password=Truxton4n6;" from TruxtonSettings.xml XML element name "truxton_options.mbconnectionstring" beginning at line 3 column 3 byte 146 of "C:\ProgramData\Truxton\Settings\TruxtonSettings.xml"
 +
solr_url : "http://localhost:8983/solr/truxton-core" from TruxtonSettings.xml XML element name "truxton_options.solr_url" beginning at line 7 column 3 byte 426 of "C:\ProgramData\Truxton\Settings\TruxtonSettings.xml"
 +
who_wrote_this : "Written by SYSTEM at 2022-07-24 08:29:10 (local time) on DESKTOP-5IFOKKS process id 3432, version 4.2.1.724, command line: "C:\Program Files\Truxton\Loader\Load.exe" --create_truxton_settings_xml Yes --dbconnectionstring "Host=localhost;Port=5432;Database=Truxton;Username=postgres;Password=Truxton4n6;" --mbconnectionstring "Host=localhost;Port=5432;Database=TruxtonMessageBus;Username=postgres;Password=Truxton4n6;" --datadir "C:\Truxton Data" --machineid "671B7D4B-0D2E-4A00-84D6-FD9F6CA1DEBA" --solr_url "http://localhost:8983/solr/truxton-core" --CreateTheDatabase 1" from TruxtonSettings.xml XML element name "truxton_options.who_wrote_this" beginning at line 53 column 3 byte 3993 of "C:\ProgramData\Truxton\Settings\TruxtonSettings.xml"
 +
 
 +
End of Options, Start of Non-Options
 +
 
 +
End of All Option Dump
 +
 
 +
2022-07-25T16:26:06Z 2CAC-84D4 : Parsing 3 command line arguments
 +
2022-07-25T16:26:06Z 2CAC-84D4 : First command line argument was something that ended in .exe, skipping it
 +
2022-07-25T16:26:06Z 2CAC-84D4 : Beginning to parse the command line arguments at array index 1
 +
2022-07-25T16:26:06Z 2CAC-84D4 : command_line_argument[ 1 ] = "-debugoptions"
 +
2022-07-25T16:26:06Z 2CAC-84D4 : command_line_argument[ 2 ] = "1" is the value of option "debugoptions"
 +
2022-07-25T16:26:06Z 2CAC-84D4 : No existing option named "debugoptions" was found, creating a new one
 +
2022-07-25T16:26:06Z 2CAC-84D4 : Finished parsing 3 command line arguments, there are now 0 NotOptions
 +
2022-07-25T16:26:06Z 2CAC-84D4 : Processing "C:\ProgramData\Truxton\Settings\TruxtonSettings.xml" as TruxtonSettings.xml(C:\ProgramData\Truxton\Settings\TruxtonSettings.xml)
 +
2022-07-25T16:26:06Z 2CAC-84D4 : Processing "C:\Program Files\Truxton\Loader\Identify.config" as the ETL.config file
 +
2022-07-25T16:26:06Z 2CAC-84D4 : Can't parse "C:\Program Files\Truxton\Loader\Identify.config" as XML
 +
2022-07-25T16:26:06Z 2CAC-84D4 : Processing "C:\ProgramData\Truxton\Settings\TruxtonSettings.machine" as the ETL.machine file
 +
2022-07-25T16:26:06Z 2CAC-84D4 : Can't parse "C:\ProgramData\Truxton\Settings\TruxtonSettings.machine" as XML
 +
2022-07-25T16:26:06Z 2CAC-84D4 : Getting Options from Truxton Database
 +
2022-07-25T16:26:06Z 2CAC-84D4 : All Option Dump (The first entry of duplicate lines is the value that will be used by the program.)
 
Process "C:\Program Files\Truxton\Loader\Identify.exe"
 
Process "C:\Program Files\Truxton\Loader\Identify.exe"
−
createthedatabase : "1" from TruxtonSettings.xml XML element name truxton_options.CreateTheDatabase beginning at line 15 column 3 byte 485 of "C:\ProgramData\Truxton\Settings\TruxtonSettings.xml"
+
createthedatabase : "1" from TruxtonSettings.xml XML element name "truxton_options.CreateTheDatabase" beginning at line 6 column 3 byte 382 of "C:\ProgramData\Truxton\Settings\TruxtonSettings.xml"
−
datadir : "C:\Truxton Data" from TruxtonSettings.xml XML element name truxton_options.datadir beginning at line 13 column 3 byte 384 of "C:\ProgramData\Truxton\Settings\TruxtonSettings.xml"
+
datadir : "C:\Truxton Data" from TruxtonSettings.xml XML element name "truxton_options.datadir" beginning at line 4 column 3 byte 281 of "C:\ProgramData\Truxton\Settings\TruxtonSettings.xml"
−
dbname : "Truxton" from TruxtonSettings.xml XML element name truxton_options.dbname beginning at line 5 column 3 byte 123 of "C:\ProgramData\Truxton\Settings\TruxtonSettings.xml"
+
dbconnectionstring : "Host=localhost;Port=5432;Database=Truxton;Username=postgres;Password=Truxton4n6;" from TruxtonSettings.xml XML element name "truxton_options.dbconnectionstring" beginning at line 2 column 3 byte 21 of "C:\ProgramData\Truxton\Settings\TruxtonSettings.xml"
−
dbpassword : "Truxton4n6" from TruxtonSettings.xml XML element name truxton_options.dbpassword beginning at line 7 column 3 byte 180 of "C:\ProgramData\Truxton\Settings\TruxtonSettings.xml"
 
−
dbport : "5432" from TruxtonSettings.xml XML element name truxton_options.dbport beginning at line 4 column 3 byte 98 of "C:\ProgramData\Truxton\Settings\TruxtonSettings.xml"
 
−
dbserver : "localhost" from TruxtonSettings.xml XML element name truxton_options.dbserver beginning at line 3 column 3 byte 64 of "C:\ProgramData\Truxton\Settings\TruxtonSettings.xml"
 
−
dbsr1 :
 
−
dbuser : "postgres" from TruxtonSettings.xml XML element name truxton_options.dbuser beginning at line 6 column 3 byte 151 of "C:\ProgramData\Truxton\Settings\TruxtonSettings.xml"
 
 
debugoptions : "1" from Command Line argument 2 (zero is the first)
 
debugoptions : "1" from Command Line argument 2 (zero is the first)
−
filegroupshavebeeninitialized : "1" from TruxtonSettings.xml XML element name truxton_options.FileGroupsHaveBeenInitialized beginning at line 17 column 3 byte 593 of "C:\ProgramData\Truxton\Settings\TruxtonSettings.xml"
+
filegroupshavebeeninitialized : "1" from TruxtonSettings.xml XML element name "truxton_options.FileGroupsHaveBeenInitialized" beginning at line 16 column 3 byte 833 of "C:\ProgramData\Truxton\Settings\TruxtonSettings.xml"
−
machineid : "64B30278-F250-4896-82EB-1AC416DB30B2" from TruxtonSettings.xml XML element name truxton_options.machineid beginning at line 14 column 3 byte 422 of "C:\ProgramData\Truxton\Settings\TruxtonSettings.xml"
+
machineid : "671B7D4B-0D2E-4A00-84D6-FD9F6CA1DEBA" from TruxtonSettings.xml XML element name "truxton_options.machineid" beginning at line 5 column 3 byte 319 of "C:\ProgramData\Truxton\Settings\TruxtonSettings.xml"
−
mqname : "TruxtonMessageBus" from TruxtonSettings.xml XML element name truxton_options.mqname beginning at line 10 column 3 byte 278 of "C:\ProgramData\Truxton\Settings\TruxtonSettings.xml"
+
mbconnectionstring : "Host=localhost;Port=5432;Database=TruxtonMessageBus;Username=postgres;Password=Truxton4n6;" from TruxtonSettings.xml XML element name "truxton_options.mbconnectionstring" beginning at line 3 column 3 byte 146 of "C:\ProgramData\Truxton\Settings\TruxtonSettings.xml"
−
mqpassword : "Truxton4n6" from TruxtonSettings.xml XML element name truxton_options.mqpassword beginning at line 12 column 3 byte 345 of "C:\ProgramData\Truxton\Settings\TruxtonSettings.xml"
+
solr_url : "http://localhost:8983/solr/truxton-core" from TruxtonSettings.xml XML element name "truxton_options.solr_url" beginning at line 7 column 3 byte 426 of "C:\ProgramData\Truxton\Settings\TruxtonSettings.xml"
−
mqport : "5432" from TruxtonSettings.xml XML element name truxton_options.mqport beginning at line 9 column 3 byte 253 of "C:\ProgramData\Truxton\Settings\TruxtonSettings.xml"
+
who_wrote_this : "Written by SYSTEM at 2022-07-24 08:29:10 (local time) on DESKTOP-5IFOKKS process id 3432, version 4.2.1.724, command line: "C:\Program Files\Truxton\Loader\Load.exe" --create_truxton_settings_xml Yes --dbconnectionstring "Host=localhost;Port=5432;Database=Truxton;Username=postgres;Password=Truxton4n6;" --mbconnectionstring "Host=localhost;Port=5432;Database=TruxtonMessageBus;Username=postgres;Password=Truxton4n6;" --datadir "C:\Truxton Data" --machineid "671B7D4B-0D2E-4A00-84D6-FD9F6CA1DEBA" --solr_url "http://localhost:8983/solr/truxton-core" --CreateTheDatabase 1" from TruxtonSettings.xml XML element name "truxton_options.who_wrote_this" beginning at line 53 column 3 byte 3993 of "C:\ProgramData\Truxton\Settings\TruxtonSettings.xml"
−
mqserver : "localhost" from TruxtonSettings.xml XML element name truxton_options.mqserver beginning at line 8 column 3 byte 219 of "C:\ProgramData\Truxton\Settings\TruxtonSettings.xml"
 
−
mquser : "postgres" from TruxtonSettings.xml XML element name truxton_options.mquser beginning at line 11 column 3 byte 316 of "C:\ProgramData\Truxton\Settings\TruxtonSettings.xml"
 
−
solr_url : "http://localhost:8983/solr/truxton-core" from TruxtonSettings.xml XML element name truxton_options.solr_url beginning at line 16 column 3 byte 529 of "C:\ProgramData\Truxton\Settings\TruxtonSettings.xml"
 
−
writesettings :
 
  
 
End of Options, Start of Non-Options
 
End of Options, Start of Non-Options
−
"identify"
 
  
 
End of All Option Dump
 
End of All Option Dump
  
−
2019-10-16T12:31:21Z 4454-6528 : Processing "C:\ProgramData\Truxton\Settings\TruxtonSettings.xml" as TruxtonSettings.xml(C:\ProgramData\Truxton\Settings\TruxtonSettings.xml)
+
2022-07-25T16:26:06Z 2CAC-84D4 : Parsing 3 command line arguments
−
2019-10-16T12:31:21Z 4454-6528 : Getting Options from Truxton Database
+
2022-07-25T16:26:06Z 2CAC-84D4 : First command line argument was something that ended in .exe, skipping it
−
2019-10-16T12:31:21Z 4454-6528 : Parsing 3 command line arguments
+
2022-07-25T16:26:06Z 2CAC-84D4 : Beginning to parse the command line arguments at array index 1
−
2019-10-16T12:31:21Z 4454-6528 : Beginning to parse the command line arguments at array index 0
+
2022-07-25T16:26:06Z 2CAC-84D4 : command_line_argument[ 1 ] = "-debugoptions"
−
2019-10-16T12:31:21Z 4454-6528 : command_line_argument[ 0 ] = "identify"
+
2022-07-25T16:26:06Z 2CAC-84D4 : command_line_argument[ 2 ] = "1" is the value of option "debugoptions"
−
2019-10-16T12:31:21Z 4454-6528 : command_line_argument[ 0 ] ("identify") is being added as a NotOption.
+
2022-07-25T16:26:06Z 2CAC-84D4 : No existing option named "debugoptions" was found, creating a new one
−
2019-10-16T12:31:21Z 4454-6528 : command_line_argument[ 0 ] ("identify") was added as a NotOption number 0.
+
2022-07-25T16:26:06Z 2CAC-84D4 : Finished parsing 3 command line arguments, there are now 0 NotOptions
−
2019-10-16T12:31:21Z 4454-6528 : command_line_argument[ 1 ] = "-debugoptions"
+
2022-07-25T16:26:06Z 2CAC-84D4 : Processing "C:\ProgramData\Truxton\Settings\TruxtonSettings.xml" as TruxtonSettings.xml(C:\ProgramData\Truxton\Settings\TruxtonSettings.xml)
−
2019-10-16T12:31:21Z 4454-6528 : command_line_argument[ 2 ] = "1" is the value of option "debugoptions"
+
2022-07-25T16:26:06Z 2CAC-84D4 : Processing "C:\Program Files\Truxton\Loader\Identify.config" as the ETL.config file
−
2019-10-16T12:31:21Z 4454-6528 : No existing option named "debugoptions" was found, creating a new one
+
2022-07-25T16:26:06Z 2CAC-84D4 : Can't parse "C:\Program Files\Truxton\Loader\Identify.config" as XML
−
2019-10-16T12:31:21Z 4454-6528 : Finished parsing 3 command line arguments, there are now 1 NotOptions
+
2022-07-25T16:26:06Z 2CAC-84D4 : Processing "C:\ProgramData\Truxton\Settings\TruxtonSettings.machine" as the ETL.machine file
−
2019-10-16T12:31:21Z 4454-6528 : Processing "C:\Program Files\Truxton\Loader\Identify.config" as the ETL.config file
+
2022-07-25T16:26:06Z 2CAC-84D4 : Can't parse "C:\ProgramData\Truxton\Settings\TruxtonSettings.machine" as XML
−
2019-10-16T12:31:21Z 4454-6528 : Can't parse "C:\Program Files\Truxton\Loader\Identify.config" as XML
+
2022-07-25T16:26:06Z 2CAC-84D4 : Getting Options from Truxton Database
−
2019-10-16T12:31:21Z 4454-6528 : Processing "C:\ProgramData\Truxton\Settings\TruxtonSettings.machine" as the ETL.machine file
+
2022-07-25T16:26:06Z 2CAC-84D4 : All Option Dump (The first entry of duplicate lines is the value that will be used by the program.)
−
2019-10-16T12:31:21Z 4454-6528 : Can't parse "C:\ProgramData\Truxton\Settings\TruxtonSettings.machine" as XML
 
−
2019-10-16T12:31:21Z 4454-6528 : All Option Dump (The first entry of duplicate lines is the value that will be used by the program.)
 
 
Process "C:\Program Files\Truxton\Loader\Identify.exe"
 
Process "C:\Program Files\Truxton\Loader\Identify.exe"
 
aci : "10007" from Default definitions
 
aci : "10007" from Default definitions
Line 212: Line 236:
 
cmid : "0" from Default definitions
 
cmid : "0" from Default definitions
 
controlledby : "0" from Default definitions
 
controlledby : "0" from Default definitions
−
createthedatabase : "1" from TruxtonSettings.xml XML element name truxton_options.CreateTheDatabase beginning at line 15 column 3 byte 485 of "C:\ProgramData\Truxton\Settings\TruxtonSettings.xml"
+
createthedatabase : "1" from TruxtonSettings.xml XML element name "truxton_options.CreateTheDatabase" beginning at line 6 column 3 byte 382 of "C:\ProgramData\Truxton\Settings\TruxtonSettings.xml"
 
cunk : "1" from Default definitions
 
cunk : "1" from Default definitions
 
cws : "10485760" from Default definitions
 
cws : "10485760" from Default definitions
−
datadir : "C:\Truxton Data" from TruxtonSettings.xml XML element name truxton_options.datadir beginning at line 13 column 3 byte 384 of "C:\ProgramData\Truxton\Settings\TruxtonSettings.xml"
+
datadir : "C:\Truxton Data" from TruxtonSettings.xml XML element name "truxton_options.datadir" beginning at line 4 column 3 byte 281 of "C:\ProgramData\Truxton\Settings\TruxtonSettings.xml"
−
dbname : "Truxton" from TruxtonSettings.xml XML element name truxton_options.dbname beginning at line 5 column 3 byte 123 of "C:\ProgramData\Truxton\Settings\TruxtonSettings.xml"
+
dbconnectionstring : "Host=localhost;Port=5432;Database=Truxton;Username=postgres;Password=Truxton4n6;" from TruxtonSettings.xml XML element name "truxton_options.dbconnectionstring" beginning at line 2 column 3 byte 21 of "C:\ProgramData\Truxton\Settings\TruxtonSettings.xml"
−
dbname : "Truxton" from Default definitions
+
dbflavor : "1" from Default definitions
−
dbpassword : "Truxton4n6" from TruxtonSettings.xml XML element name truxton_options.dbpassword beginning at line 7 column 3 byte 180 of "C:\ProgramData\Truxton\Settings\TruxtonSettings.xml"
 
−
dbport : "5432" from TruxtonSettings.xml XML element name truxton_options.dbport beginning at line 4 column 3 byte 98 of "C:\ProgramData\Truxton\Settings\TruxtonSettings.xml"
 
−
dbport : "5432" from Default definitions
 
−
dbserver : "localhost" from TruxtonSettings.xml XML element name truxton_options.dbserver beginning at line 3 column 3 byte 64 of "C:\ProgramData\Truxton\Settings\TruxtonSettings.xml"
 
−
dbserver : "127.0.0.1" from Default definitions
 
−
dbsr1 :
 
−
dbuser : "postgres" from TruxtonSettings.xml XML element name truxton_options.dbuser beginning at line 6 column 3 byte 151 of "C:\ProgramData\Truxton\Settings\TruxtonSettings.xml"
 
−
dbuser : "postgres" from Default definitions
 
 
debugfileidmapping : "0" from Default definitions
 
debugfileidmapping : "0" from Default definitions
 
debugoptions : "1" from Command Line argument 2 (zero is the first)
 
debugoptions : "1" from Command Line argument 2 (zero is the first)
Line 231: Line 247:
 
deepsleep : "45007" from Default definitions
 
deepsleep : "45007" from Default definitions
 
deletedfiles : "1" from Default definitions
 
deletedfiles : "1" from Default definitions
 +
depotmaxsize : "1099511627776" from Default definitions
 
depotpagesize : "4096" from Default definitions
 
depotpagesize : "4096" from Default definitions
 
ds : "0" from Default definitions
 
ds : "0" from Default definitions
Line 236: Line 253:
 
excludefilesbytype : "0" from Default definitions
 
excludefilesbytype : "0" from Default definitions
 
excludes : "0" from Default definitions
 
excludes : "0" from Default definitions
−
fct : "300" from Default definitions
+
fct : "306" from Default definitions
−
filegroupshavebeeninitialized : "1" from TruxtonSettings.xml XML element name truxton_options.FileGroupsHaveBeenInitialized beginning at line 17 column 3 byte 593 of "C:\ProgramData\Truxton\Settings\TruxtonSettings.xml"
+
filegroupshavebeeninitialized : "1" from TruxtonSettings.xml XML element name "truxton_options.FileGroupsHaveBeenInitialized" beginning at line 16 column 3 byte 833 of "C:\ProgramData\Truxton\Settings\TruxtonSettings.xml"
 
fileslack : "1" from Default definitions
 
fileslack : "1" from Default definitions
 
fqhi : "7500" from Default definitions
 
fqhi : "7500" from Default definitions
Line 250: Line 267:
 
itype : "0" from Default definitions
 
itype : "0" from Default definitions
 
loadas : "0" from Default definitions
 
loadas : "0" from Default definitions
−
machineid : "64B30278-F250-4896-82EB-1AC416DB30B2" from TruxtonSettings.xml XML element name truxton_options.machineid beginning at line 14 column 3 byte 422 of "C:\ProgramData\Truxton\Settings\TruxtonSettings.xml"
+
lq : "load" from Default definitions
 +
machineid : "671B7D4B-0D2E-4A00-84D6-FD9F6CA1DEBA" from TruxtonSettings.xml XML element name "truxton_options.machineid" beginning at line 5 column 3 byte 319 of "C:\ProgramData\Truxton\Settings\TruxtonSettings.xml"
 
maxsleep : "600101" from Default definitions
 
maxsleep : "600101" from Default definitions
 +
mbconnectionstring : "Host=localhost;Port=5432;Database=TruxtonMessageBus;Username=postgres;Password=Truxton4n6;" from TruxtonSettings.xml XML element name "truxton_options.mbconnectionstring" beginning at line 3 column 3 byte 146 of "C:\ProgramData\Truxton\Settings\TruxtonSettings.xml"
 
metacarve : "1" from Default definitions
 
metacarve : "1" from Default definitions
 
minsleep : "3011" from Default definitions
 
minsleep : "3011" from Default definitions
 
mlat : "0" from Default definitions
 
mlat : "0" from Default definitions
 
mlong : "0" from Default definitions
 
mlong : "0" from Default definitions
−
mqname : "TruxtonMessageBus" from TruxtonSettings.xml XML element name truxton_options.mqname beginning at line 10 column 3 byte 278 of "C:\ProgramData\Truxton\Settings\TruxtonSettings.xml"
+
mqconcentrator : "0" from Default definitions
−
mqname : "TruxtonMessageBus" from Default definitions
 
−
mqpassword : "Truxton4n6" from TruxtonSettings.xml XML element name truxton_options.mqpassword beginning at line 12 column 3 byte 345 of "C:\ProgramData\Truxton\Settings\TruxtonSettings.xml"
 
−
mqport : "5432" from TruxtonSettings.xml XML element name truxton_options.mqport beginning at line 9 column 3 byte 253 of "C:\ProgramData\Truxton\Settings\TruxtonSettings.xml"
 
−
mqport : "0" from Default definitions
 
−
mqserver : "localhost" from TruxtonSettings.xml XML element name truxton_options.mqserver beginning at line 8 column 3 byte 219 of "C:\ProgramData\Truxton\Settings\TruxtonSettings.xml"
 
−
mqserver : "127.0.0.1" from Default definitions
 
−
mquser : "postgres" from TruxtonSettings.xml XML element name truxton_options.mquser beginning at line 11 column 3 byte 316 of "C:\ProgramData\Truxton\Settings\TruxtonSettings.xml"
 
−
mquser : "postgres" from Default definitions
 
 
mr : "0" from Default definitions
 
mr : "0" from Default definitions
 +
mt : "0" from Default definitions
 
nocontents : "0" from Default definitions
 
nocontents : "0" from Default definitions
 
nohashsets : "0" from Default definitions
 
nohashsets : "0" from Default definitions
Line 282: Line 294:
 
skipdupmedia : "0" from Default definitions
 
skipdupmedia : "0" from Default definitions
 
smtp_server : "localhost" from Default definitions
 
smtp_server : "localhost" from Default definitions
−
solr_url : "http://localhost:8983/solr/truxton-core" from TruxtonSettings.xml XML element name truxton_options.solr_url beginning at line 16 column 3 byte 529 of "C:\ProgramData\Truxton\Settings\TruxtonSettings.xml"
+
solr_url : "http://localhost:8983/solr/truxton-core" from TruxtonSettings.xml XML element name "truxton_options.solr_url" beginning at line 7 column 3 byte 426 of "C:\ProgramData\Truxton\Settings\TruxtonSettings.xml"
 
startatmft : "0" from Default definitions
 
startatmft : "0" from Default definitions
 
syslogport : "514" from Default definitions
 
syslogport : "514" from Default definitions
Line 292: Line 304:
 
verbose : "0" from Default definitions
 
verbose : "0" from Default definitions
 
wait : "none" from Default definitions
 
wait : "none" from Default definitions
−
writesettings :  
+
who_wrote_this : "Written by SYSTEM at 2022-07-24 08:29:10 (local time) on DESKTOP-5IFOKKS process id 3432, version 4.2.1.724, command line: "C:\Program Files\Truxton\Loader\Load.exe" --create_truxton_settings_xml Yes --dbconnectionstring "Host=localhost;Port=5432;Database=Truxton;Username=postgres;Password=Truxton4n6;" --mbconnectionstring "Host=localhost;Port=5432;Database=TruxtonMessageBus;Username=postgres;Password=Truxton4n6;" --datadir "C:\Truxton Data" --machineid "671B7D4B-0D2E-4A00-84D6-FD9F6CA1DEBA" --solr_url "http://localhost:8983/solr/truxton-core" --CreateTheDatabase 1" from TruxtonSettings.xml XML element name "truxton_options.who_wrote_this" beginning at line 53 column 3 byte 3993 of "C:\ProgramData\Truxton\Settings\TruxtonSettings.xml"
 
xorcarve : "1" from Default definitions
 
xorcarve : "1" from Default definitions
  
 
End of Options, Start of Non-Options
 
End of Options, Start of Non-Options
−
"identify"
 
  
 
End of All Option Dump
 
End of All Option Dump
 +
</pre>
 +
 +
=Data Types=
 +
There are only three types of data for options.
 +
 +
==String==
 +
String data is any series of characters.
  
−
2019-10-16T12:31:21Z 4454-6528 : application_name='Truxton Identify' dbname='Truxton' user='postgres' password='Truxton4n6' host='localhost' port=5432
+
==Integer==
−
2019-10-16T12:31:21Z 4454-6528 : application_name='Truxton Identify' dbname='TruxtonMessageBus' user='postgres' password='Truxton4n6' host='localhost' port=5432
+
The string data is interpreted as a sixty-four bit unsigned integer.
−
2019-10-16T12:31:21Z 4454-6528 : Truxton Identify can't get connection string variable dbserver variable "logserver"
+
The option system will automatically convert hexadecimal strings for you if they begin with <code>x</code> or <code>0x</code>.
−
2019-10-16T12:31:21Z 4454-6528 : application_name='Truxton Identify' dbname='Truxton' user='postgres' password='Truxton4n6' host='localhost' port=5432
+
For example, of you have an integer field specified as <code>x2a</code> and read it as an integer value, 42 will be returned.
−
</pre>
 
  
−
=Random Notes=
+
==Boolean==
−
Lessons learned from the field...
+
Booleans are either true or false.
 +
The string data will be interpreted according to these rules:
 +
* If the string value is <code>true</code> or <code>yes</code>, the returned value will be true.
 +
* If the string value interpreted as an integer is not zero, the returned value will be true.
 +
* Return value is false in all other cases.
  
 
==ReFS Deduplication==
 
==ReFS Deduplication==
 
It looks like Windows Storage Spaces has limitations on [https://docs.microsoft.com/en-us/windows-server/storage/data-deduplication/whats-new deduplication] of volumes.
 
It looks like Windows Storage Spaces has limitations on [https://docs.microsoft.com/en-us/windows-server/storage/data-deduplication/whats-new deduplication] of volumes.
 
It appears that the maximum volume size is 64TB while only the first 4TB of a file will be deduplicated.
 
It appears that the maximum volume size is 64TB while only the first 4TB of a file will be deduplicated.
 +
 +
=Configuration File Format=
 +
Truxton uses XML as the format for configuration.
 +
However, it uses it differently than most applications.
 +
Truxton will parse the specified XML file, then enumerate through all of the elements with a name of <code>truxton_options</code> and use only those sections of the file for configuration.
 +
This allows you to insert Truxton configuration into any XML file.
 +
 +
==Adding a Setting==
 +
Here's a basic <code>[[#Truxton Settings File|TruxtonSettings.xml]]</code> file:
 +
<source lang="XML">
 +
<truxton_options>
 +
  <dbconnectionstring>Host=localhost;Port=5432;Database=Truxton;Username=postgres;Password=Password123;</dbconnectionstring>
 +
  <mbconnectionstring>Host=localhost;Port=5432;Database=TruxtonMessageBus;Username=postgres;Password=Password123;</mbconnectionstring>
 +
  <datadir>C:\Truxton Data</datadir>
 +
  <tempdir>C:\Truxton Data\Temp\</tempdir>
 +
  <sqlite_exploitation_scripts_folder>C:\Program Files\Truxton\Loader\SQLite Exploitation Scripts</sqlite_exploitation_scripts_folder>
 +
  <machineid>9D6D69B5-2568-4CEC-B4D4-1A4B56FBA557</machineid>
 +
  <solr_url>http://localhost:8983/solr/truxton-core</solr_url>
 +
  <tesseract>C:\Program Files\Tesseract-OCR\tesseract.exe</tesseract>
 +
</truxton_options>
 +
</source>
 +
 +
We want to reduce our storage by saving a file's contents only once throughout all of Truxton.
 +
This is controlled by the <code>globaldedupe</code> option.
 +
We would edit the <code>[[#Truxton Settings File|TruxtonSettings.xml]]</code> file and add a new element to the <code>truxton_options</code> section.
 +
 +
<source lang="XML" highlight="2">
 +
<truxton_options>
 +
  <globaldedupe>1</globaldedupe>
 +
  <dbconnectionstring>Host=localhost;Port=5432;Database=Truxton;Username=postgres;Password=Password123;</dbconnectionstring>
 +
  <mbconnectionstring>Host=localhost;Port=5432;Database=TruxtonMessageBus;Username=postgres;Password=Password123;</mbconnectionstring>
 +
  <datadir>C:\Truxton Data</datadir>
 +
  <tempdir>C:\Truxton Data\Temp\</tempdir>
 +
  <sqlite_exploitation_scripts_folder>C:\Program Files\Truxton\Loader\SQLite Exploitation Scripts</sqlite_exploitation_scripts_folder>
 +
  <machineid>9D6D69B5-2568-4CEC-B4D4-1A4B56FBA557</machineid>
 +
  <solr_url>http://localhost:8983/solr/truxton-core</solr_url>
 +
  <tesseract>C:\Program Files\Tesseract-OCR\tesseract.exe</tesseract>
 +
</truxton_options>
 +
</source>
 +
 +
Once you save the file, you must restart the [[Truxton Service|Truxton service]] in order for the new settings to take affect.

Latest revision as of 06:12, 16 April 2026

Description

The way to communicate settings to Truxton is through a class called Options. This is an interpretation of the Ye Olde "Command Line Options" of which much has been written. In Truxton, an option is prefaced by a name followed by the value. There is only one value per option. In this article, we may refer to an option as a parameter or setting.

Sources

The following are the nine different places that a parameter can come from. The lower the number, the greater the precedence that option has. The harder it is for a user to define an option, the greater weight that option will be given. It is harder for a user to specify an option on a command line because they must type it out. The user exerts no effort whatsoever to use a default value compiled into programs. Therefore, the command line option will be used instead of the compiled default.

  1. Command Line Parameters - Highest priority
  2. Command Line Configuration File
  3. Command Line Database
  4. Environment Variable
  5. ETL Configuration File
  6. ETL Machine File
  7. Database by Machine
  8. Truxton Settings File
  9. Compiled Defaults - Lowest priority

Why So Many Choices?

There are many ways to run processes. So many scenarios require different ways to configure. If you host an ETL process in a Docker image, it is far easier to insert environment variables into the deployment instead of pushing configuration files into the image. This allows you to create configuration-free Docker images containing the ETL processes you want. Writing a one-time script to process something in Truxton can use command line arguments for customization instead of editing configuration files.

Truxton tries to make it possible to configure the processes however you my need to run them.

Command Line Parameters

If a human takes the time to physically type parameters on a command line, they must really mean that they want this value to be used. Anything specified on a command line takes precedence over any other location. The format of command line parameters is parameter name followed by a space and the parameter value. 

load.exe -w 1 --xx abc /z "3 4 and 5" disk.dat

In the example above there are three options specified and one non-option.

Option Name Option Value
w 1
xx abc
z 3 4 and 5

The non-option is "disk.dat" A command line is the only way to specify a non-option. Option names can be prefaced with a single dash, double dash or a slash depending on your command line religion.

Command Line Configuration File

You can put the name of a configuration file on the command line. 

Identify.exe my_settings.xml

The my_settings.xml would be assumed to be a file name since it was not part of an option. That file would then be opened, parsed as XML and anything found in a truxton_options section will be used as configuration options. All XML element names used in Truxton are lower case.

The settings would look like this: 

<truxton_options>
  <w>1</w>
  <xx>abc</xx>
  <z>3 4 and 5</z>
</truxton_options>

Command Line Database

If you specify a settings identifier on the command line, Truxton will retrieve that setting from the database and use it. The GUID you specify should match the [ID] column of the [Setting] table. If a record is found, it will then parse the XML contents of the [Value] column. 

Identify.exe 51256fb4-c7f6-327a-2703-325e219ecc2b

If you actually use a random value you will have no end of mistakes. We suggest making fake ids for settings and using those. It will make typing it in far easier. Here's a fake setting that can be easily typed and remembered by a human. 

Identify.exe 12345678-1234-1234-1234-123456789abc

Environment Variable

Configuration options can also be set using environment variables. These variables must have names that begin with Truxton_ For example: 

set Truxton_w=1
set Truxton_xx=abc
set Truxton_z="3 4 and 5"
Load.exe disk.dat

ETL Configuration File

Another location is an XML configuration file specifically for this type of ETL process. Truxton will look for a file in the same directory as the executable with config as the extension. For example, the Identify.exe ETL process will look for a file named Identify.config and parse it if found.

ETL Machine File

This is a configuration file with customized settings valid for the machine that runs the ETL processes. Truxton will look for a file in the same directory as the TruxtonSettings.xml file named TruxtonSettings.machine as the extension.

Machine configuration is usually used for specifying common settings that are unique to a machine. For example, every ETL process needs some sort of temporary directory. Some of your ETL machines may have RAM disks, others may have hard drives with differing drive letters. Machine configuration files allow you to customize where the temp folder is for each machine.

Database by Machine

Each installation of Truxton has a globally unique machine identifier. This can be found in the TruxtonObject#machineid:_str element of the TruxtonSettings.XML file. Truxton will connect to the database and query the [Setting] table for a record with the machine identifier in the [ID] column. If found, the XML in the Value column will be parsed for options.

Truxton Settings File

Each installation has a TruxtonSettings.xml file. It is found in the %programdata%\Truxton\Settings folder which usually expands to C:\ProgramData\Truxton\Settings

NOTE: The ProgramData folder is normally hidden and won't show up in directory listings (or File Explorer).

Its primary purpose is to store the globally unique identifier for the machine which was generated during installation. The default database connection parameters are also stored here.

As you can see, the TruxtonSettings.xml file has very low priority.

Compiled Defaults

Each ETL member can have default options built into it. If the option is not specified anywhere above, whatever was compiled into the executable will be used. This is the last resort.

Debugging the Options

With nine different places to get options from, how could anything go wrong? If you need help debugging why an option isn't getting the value you want, use the debugoptions option. 

Identify.exe -debugoptions 1

This will cause an installation log to be created and the options written to it. This log file is found in the Truxton\OptionsDebugging directory under the %programdata% folder (which usually expands to C:\ProgramData) folder. An installation log is the only log that can be created safely if everything else fails.

Sample Options Log

Log file created 2022-07-25 12:26:07 by "C:\Program Files\Truxton\Loader\Identify.exe" running as Sam on the machine named DESKTOP-5IFOKKS (192.168.1.88).
2022-07-25T16:26:06Z 2CAC-84D4 : Parsing 3 command line arguments
2022-07-25T16:26:06Z 2CAC-84D4 : First command line argument was something that ended in .exe, skipping it
2022-07-25T16:26:06Z 2CAC-84D4 : Beginning to parse the command line arguments at array index 1
2022-07-25T16:26:06Z 2CAC-84D4 : command_line_argument[ 1 ] = "-debugoptions"
2022-07-25T16:26:06Z 2CAC-84D4 : command_line_argument[ 2 ] = "1" is the value of option "debugoptions"
2022-07-25T16:26:06Z 2CAC-84D4 : No existing option named "debugoptions" was found, creating a new one
2022-07-25T16:26:06Z 2CAC-84D4 : Finished parsing 3 command line arguments, there are now 0 NotOptions
2022-07-25T16:26:06Z 2CAC-84D4 : Processing "C:\ProgramData\Truxton\Settings\TruxtonSettings.xml" as TruxtonSettings.xml(C:\ProgramData\Truxton\Settings\TruxtonSettings.xml)
2022-07-25T16:26:06Z 2CAC-84D4 : Processing "C:\Program Files\Truxton\Loader\Identify.config" as the ETL.config file
2022-07-25T16:26:06Z 2CAC-84D4 : Can't parse "C:\Program Files\Truxton\Loader\Identify.config" as XML
2022-07-25T16:26:06Z 2CAC-84D4 : Processing "C:\ProgramData\Truxton\Settings\TruxtonSettings.machine" as the ETL.machine file
2022-07-25T16:26:06Z 2CAC-84D4 : Can't parse "C:\ProgramData\Truxton\Settings\TruxtonSettings.machine" as XML
2022-07-25T16:26:06Z 2CAC-84D4 : Getting Options from Truxton Database
2022-07-25T16:26:06Z 2CAC-84D4 : All Option Dump (The first entry of duplicate lines is the value that will be used by the program.)
Process "C:\Program Files\Truxton\Loader\Identify.exe"
createthedatabase : "1" from TruxtonSettings.xml XML element name "truxton_options.CreateTheDatabase" beginning at line 6 column 3 byte 382 of "C:\ProgramData\Truxton\Settings\TruxtonSettings.xml"
datadir : "C:\Truxton Data" from TruxtonSettings.xml XML element name "truxton_options.datadir" beginning at line 4 column 3 byte 281 of "C:\ProgramData\Truxton\Settings\TruxtonSettings.xml"
dbconnectionstring : "Host=localhost;Port=5432;Database=Truxton;Username=postgres;Password=Truxton4n6;" from TruxtonSettings.xml XML element name "truxton_options.dbconnectionstring" beginning at line 2 column 3 byte 21 of "C:\ProgramData\Truxton\Settings\TruxtonSettings.xml"
debugoptions : "1" from Command Line argument 2 (zero is the first)
filegroupshavebeeninitialized : "1" from TruxtonSettings.xml XML element name "truxton_options.FileGroupsHaveBeenInitialized" beginning at line 16 column 3 byte 833 of "C:\ProgramData\Truxton\Settings\TruxtonSettings.xml"
machineid : "671B7D4B-0D2E-4A00-84D6-FD9F6CA1DEBA" from TruxtonSettings.xml XML element name "truxton_options.machineid" beginning at line 5 column 3 byte 319 of "C:\ProgramData\Truxton\Settings\TruxtonSettings.xml"
mbconnectionstring : "Host=localhost;Port=5432;Database=TruxtonMessageBus;Username=postgres;Password=Truxton4n6;" from TruxtonSettings.xml XML element name "truxton_options.mbconnectionstring" beginning at line 3 column 3 byte 146 of "C:\ProgramData\Truxton\Settings\TruxtonSettings.xml"
solr_url : "http://localhost:8983/solr/truxton-core" from TruxtonSettings.xml XML element name "truxton_options.solr_url" beginning at line 7 column 3 byte 426 of "C:\ProgramData\Truxton\Settings\TruxtonSettings.xml"
who_wrote_this : "Written by SYSTEM at 2022-07-24 08:29:10 (local time) on DESKTOP-5IFOKKS process id 3432, version 4.2.1.724, command line: "C:\Program Files\Truxton\Loader\Load.exe" --create_truxton_settings_xml Yes --dbconnectionstring "Host=localhost;Port=5432;Database=Truxton;Username=postgres;Password=Truxton4n6;" --mbconnectionstring "Host=localhost;Port=5432;Database=TruxtonMessageBus;Username=postgres;Password=Truxton4n6;" --datadir "C:\Truxton Data" --machineid "671B7D4B-0D2E-4A00-84D6-FD9F6CA1DEBA" --solr_url "http://localhost:8983/solr/truxton-core" --CreateTheDatabase 1" from TruxtonSettings.xml XML element name "truxton_options.who_wrote_this" beginning at line 53 column 3 byte 3993 of "C:\ProgramData\Truxton\Settings\TruxtonSettings.xml"

End of Options, Start of Non-Options

End of All Option Dump

2022-07-25T16:26:06Z 2CAC-84D4 : Parsing 3 command line arguments
2022-07-25T16:26:06Z 2CAC-84D4 : First command line argument was something that ended in .exe, skipping it
2022-07-25T16:26:06Z 2CAC-84D4 : Beginning to parse the command line arguments at array index 1
2022-07-25T16:26:06Z 2CAC-84D4 : command_line_argument[ 1 ] = "-debugoptions"
2022-07-25T16:26:06Z 2CAC-84D4 : command_line_argument[ 2 ] = "1" is the value of option "debugoptions"
2022-07-25T16:26:06Z 2CAC-84D4 : No existing option named "debugoptions" was found, creating a new one
2022-07-25T16:26:06Z 2CAC-84D4 : Finished parsing 3 command line arguments, there are now 0 NotOptions
2022-07-25T16:26:06Z 2CAC-84D4 : Processing "C:\ProgramData\Truxton\Settings\TruxtonSettings.xml" as TruxtonSettings.xml(C:\ProgramData\Truxton\Settings\TruxtonSettings.xml)
2022-07-25T16:26:06Z 2CAC-84D4 : Processing "C:\Program Files\Truxton\Loader\Identify.config" as the ETL.config file
2022-07-25T16:26:06Z 2CAC-84D4 : Can't parse "C:\Program Files\Truxton\Loader\Identify.config" as XML
2022-07-25T16:26:06Z 2CAC-84D4 : Processing "C:\ProgramData\Truxton\Settings\TruxtonSettings.machine" as the ETL.machine file
2022-07-25T16:26:06Z 2CAC-84D4 : Can't parse "C:\ProgramData\Truxton\Settings\TruxtonSettings.machine" as XML
2022-07-25T16:26:06Z 2CAC-84D4 : Getting Options from Truxton Database
2022-07-25T16:26:06Z 2CAC-84D4 : All Option Dump (The first entry of duplicate lines is the value that will be used by the program.)
Process "C:\Program Files\Truxton\Loader\Identify.exe"
createthedatabase : "1" from TruxtonSettings.xml XML element name "truxton_options.CreateTheDatabase" beginning at line 6 column 3 byte 382 of "C:\ProgramData\Truxton\Settings\TruxtonSettings.xml"
datadir : "C:\Truxton Data" from TruxtonSettings.xml XML element name "truxton_options.datadir" beginning at line 4 column 3 byte 281 of "C:\ProgramData\Truxton\Settings\TruxtonSettings.xml"
dbconnectionstring : "Host=localhost;Port=5432;Database=Truxton;Username=postgres;Password=Truxton4n6;" from TruxtonSettings.xml XML element name "truxton_options.dbconnectionstring" beginning at line 2 column 3 byte 21 of "C:\ProgramData\Truxton\Settings\TruxtonSettings.xml"
debugoptions : "1" from Command Line argument 2 (zero is the first)
filegroupshavebeeninitialized : "1" from TruxtonSettings.xml XML element name "truxton_options.FileGroupsHaveBeenInitialized" beginning at line 16 column 3 byte 833 of "C:\ProgramData\Truxton\Settings\TruxtonSettings.xml"
machineid : "671B7D4B-0D2E-4A00-84D6-FD9F6CA1DEBA" from TruxtonSettings.xml XML element name "truxton_options.machineid" beginning at line 5 column 3 byte 319 of "C:\ProgramData\Truxton\Settings\TruxtonSettings.xml"
mbconnectionstring : "Host=localhost;Port=5432;Database=TruxtonMessageBus;Username=postgres;Password=Truxton4n6;" from TruxtonSettings.xml XML element name "truxton_options.mbconnectionstring" beginning at line 3 column 3 byte 146 of "C:\ProgramData\Truxton\Settings\TruxtonSettings.xml"
solr_url : "http://localhost:8983/solr/truxton-core" from TruxtonSettings.xml XML element name "truxton_options.solr_url" beginning at line 7 column 3 byte 426 of "C:\ProgramData\Truxton\Settings\TruxtonSettings.xml"
who_wrote_this : "Written by SYSTEM at 2022-07-24 08:29:10 (local time) on DESKTOP-5IFOKKS process id 3432, version 4.2.1.724, command line: "C:\Program Files\Truxton\Loader\Load.exe" --create_truxton_settings_xml Yes --dbconnectionstring "Host=localhost;Port=5432;Database=Truxton;Username=postgres;Password=Truxton4n6;" --mbconnectionstring "Host=localhost;Port=5432;Database=TruxtonMessageBus;Username=postgres;Password=Truxton4n6;" --datadir "C:\Truxton Data" --machineid "671B7D4B-0D2E-4A00-84D6-FD9F6CA1DEBA" --solr_url "http://localhost:8983/solr/truxton-core" --CreateTheDatabase 1" from TruxtonSettings.xml XML element name "truxton_options.who_wrote_this" beginning at line 53 column 3 byte 3993 of "C:\ProgramData\Truxton\Settings\TruxtonSettings.xml"

End of Options, Start of Non-Options

End of All Option Dump

2022-07-25T16:26:06Z 2CAC-84D4 : Parsing 3 command line arguments
2022-07-25T16:26:06Z 2CAC-84D4 : First command line argument was something that ended in .exe, skipping it
2022-07-25T16:26:06Z 2CAC-84D4 : Beginning to parse the command line arguments at array index 1
2022-07-25T16:26:06Z 2CAC-84D4 : command_line_argument[ 1 ] = "-debugoptions"
2022-07-25T16:26:06Z 2CAC-84D4 : command_line_argument[ 2 ] = "1" is the value of option "debugoptions"
2022-07-25T16:26:06Z 2CAC-84D4 : No existing option named "debugoptions" was found, creating a new one
2022-07-25T16:26:06Z 2CAC-84D4 : Finished parsing 3 command line arguments, there are now 0 NotOptions
2022-07-25T16:26:06Z 2CAC-84D4 : Processing "C:\ProgramData\Truxton\Settings\TruxtonSettings.xml" as TruxtonSettings.xml(C:\ProgramData\Truxton\Settings\TruxtonSettings.xml)
2022-07-25T16:26:06Z 2CAC-84D4 : Processing "C:\Program Files\Truxton\Loader\Identify.config" as the ETL.config file
2022-07-25T16:26:06Z 2CAC-84D4 : Can't parse "C:\Program Files\Truxton\Loader\Identify.config" as XML
2022-07-25T16:26:06Z 2CAC-84D4 : Processing "C:\ProgramData\Truxton\Settings\TruxtonSettings.machine" as the ETL.machine file
2022-07-25T16:26:06Z 2CAC-84D4 : Can't parse "C:\ProgramData\Truxton\Settings\TruxtonSettings.machine" as XML
2022-07-25T16:26:06Z 2CAC-84D4 : Getting Options from Truxton Database
2022-07-25T16:26:06Z 2CAC-84D4 : All Option Dump (The first entry of duplicate lines is the value that will be used by the program.)
Process "C:\Program Files\Truxton\Loader\Identify.exe"
aci : "10007" from Default definitions
alert_email_from_address : "Truxton@localhost" from Default definitions
alert_email_subject : "[Truxton Alert]" from Default definitions
carve : "1" from Default definitions
carve_threads : "1" from Default definitions
carvemode : "0" from Default definitions
carveoffset : "65536" from Default definitions
carveunknown : "0" from Default definitions
cmid : "0" from Default definitions
controlledby : "0" from Default definitions
createthedatabase : "1" from TruxtonSettings.xml XML element name "truxton_options.CreateTheDatabase" beginning at line 6 column 3 byte 382 of "C:\ProgramData\Truxton\Settings\TruxtonSettings.xml"
cunk : "1" from Default definitions
cws : "10485760" from Default definitions
datadir : "C:\Truxton Data" from TruxtonSettings.xml XML element name "truxton_options.datadir" beginning at line 4 column 3 byte 281 of "C:\ProgramData\Truxton\Settings\TruxtonSettings.xml"
dbconnectionstring : "Host=localhost;Port=5432;Database=Truxton;Username=postgres;Password=Truxton4n6;" from TruxtonSettings.xml XML element name "truxton_options.dbconnectionstring" beginning at line 2 column 3 byte 21 of "C:\ProgramData\Truxton\Settings\TruxtonSettings.xml"
dbflavor : "1" from Default definitions
debugfileidmapping : "0" from Default definitions
debugoptions : "1" from Command Line argument 2 (zero is the first)
dedupemessages : "0" from Default definitions
deepsleep : "45007" from Default definitions
deletedfiles : "1" from Default definitions
depotmaxsize : "1099511627776" from Default definitions
depotpagesize : "4096" from Default definitions
ds : "0" from Default definitions
ee : "0" from Default definitions
excludefilesbytype : "0" from Default definitions
excludes : "0" from Default definitions
fct : "306" from Default definitions
filegroupshavebeeninitialized : "1" from TruxtonSettings.xml XML element name "truxton_options.FileGroupsHaveBeenInitialized" beginning at line 16 column 3 byte 833 of "C:\ProgramData\Truxton\Settings\TruxtonSettings.xml"
fileslack : "1" from Default definitions
fqhi : "7500" from Default definitions
fqlo : "100" from Default definitions
hashsetall : "0" from Default definitions
iei : "1" from Default definitions
imagelength : "-1" from Default definitions
imageoffset : "0" from Default definitions
includefilesbytype : "0" from Default definitions
includes : "0" from Default definitions
istatus : "1" from Default definitions
itype : "0" from Default definitions
loadas : "0" from Default definitions
lq : "load" from Default definitions
machineid : "671B7D4B-0D2E-4A00-84D6-FD9F6CA1DEBA" from TruxtonSettings.xml XML element name "truxton_options.machineid" beginning at line 5 column 3 byte 319 of "C:\ProgramData\Truxton\Settings\TruxtonSettings.xml"
maxsleep : "600101" from Default definitions
mbconnectionstring : "Host=localhost;Port=5432;Database=TruxtonMessageBus;Username=postgres;Password=Truxton4n6;" from TruxtonSettings.xml XML element name "truxton_options.mbconnectionstring" beginning at line 3 column 3 byte 146 of "C:\ProgramData\Truxton\Settings\TruxtonSettings.xml"
metacarve : "1" from Default definitions
minsleep : "3011" from Default definitions
mlat : "0" from Default definitions
mlong : "0" from Default definitions
mqconcentrator : "0" from Default definitions
mr : "0" from Default definitions
mt : "0" from Default definitions
nocontents : "0" from Default definitions
nohashsets : "0" from Default definitions
norouting : "0" from Default definitions
nostartdelay : "0" from Default definitions
nostore : "0" from Default definitions
numberofbuffers : "2048" from Default definitions
numberofbytesperbuffer : "65536" from Default definitions
numberoffiles : "9223372036854775807" from Default definitions
pfiles : "1" from Default definitions
pfree : "1" from Default definitions
priority : "1000" from Default definitions
rapid : "0" from Default definitions
recovery : "0" from Default definitions
reprocess : "0" from Default definitions
skipdupmedia : "0" from Default definitions
smtp_server : "localhost" from Default definitions
solr_url : "http://localhost:8983/solr/truxton-core" from TruxtonSettings.xml XML element name "truxton_options.solr_url" beginning at line 7 column 3 byte 426 of "C:\ProgramData\Truxton\Settings\TruxtonSettings.xml"
startatmft : "0" from Default definitions
syslogport : "514" from Default definitions
timeout : "120" from Default definitions
triage : "0" from Default definitions
tsk_verbose : "0" from Default definitions
updatemedianumberofchildren : "1" from Default definitions
updatepathid : "1" from Default definitions
verbose : "0" from Default definitions
wait : "none" from Default definitions
who_wrote_this : "Written by SYSTEM at 2022-07-24 08:29:10 (local time) on DESKTOP-5IFOKKS process id 3432, version 4.2.1.724, command line: "C:\Program Files\Truxton\Loader\Load.exe" --create_truxton_settings_xml Yes --dbconnectionstring "Host=localhost;Port=5432;Database=Truxton;Username=postgres;Password=Truxton4n6;" --mbconnectionstring "Host=localhost;Port=5432;Database=TruxtonMessageBus;Username=postgres;Password=Truxton4n6;" --datadir "C:\Truxton Data" --machineid "671B7D4B-0D2E-4A00-84D6-FD9F6CA1DEBA" --solr_url "http://localhost:8983/solr/truxton-core" --CreateTheDatabase 1" from TruxtonSettings.xml XML element name "truxton_options.who_wrote_this" beginning at line 53 column 3 byte 3993 of "C:\ProgramData\Truxton\Settings\TruxtonSettings.xml"
xorcarve : "1" from Default definitions

End of Options, Start of Non-Options

End of All Option Dump

Data Types

There are only three types of data for options.

String

String data is any series of characters.

Integer

The string data is interpreted as a sixty-four bit unsigned integer. The option system will automatically convert hexadecimal strings for you if they begin with x or 0x. For example, of you have an integer field specified as x2a and read it as an integer value, 42 will be returned.

Boolean

Booleans are either true or false. The string data will be interpreted according to these rules:

  • If the string value is true or yes, the returned value will be true.
  • If the string value interpreted as an integer is not zero, the returned value will be true.
  • Return value is false in all other cases.

ReFS Deduplication

It looks like Windows Storage Spaces has limitations on deduplication of volumes. It appears that the maximum volume size is 64TB while only the first 4TB of a file will be deduplicated.

Configuration File Format

Truxton uses XML as the format for configuration. However, it uses it differently than most applications. Truxton will parse the specified XML file, then enumerate through all of the elements with a name of truxton_options and use only those sections of the file for configuration. This allows you to insert Truxton configuration into any XML file.

Adding a Setting

Here's a basic TruxtonSettings.xml file: 

<truxton_options>
  <dbconnectionstring>Host=localhost;Port=5432;Database=Truxton;Username=postgres;Password=Password123;</dbconnectionstring>
  <mbconnectionstring>Host=localhost;Port=5432;Database=TruxtonMessageBus;Username=postgres;Password=Password123;</mbconnectionstring>
  <datadir>C:\Truxton Data</datadir>
  <tempdir>C:\Truxton Data\Temp\</tempdir>
  <sqlite_exploitation_scripts_folder>C:\Program Files\Truxton\Loader\SQLite Exploitation Scripts</sqlite_exploitation_scripts_folder>
  <machineid>9D6D69B5-2568-4CEC-B4D4-1A4B56FBA557</machineid>
  <solr_url>http://localhost:8983/solr/truxton-core</solr_url>
  <tesseract>C:\Program Files\Tesseract-OCR\tesseract.exe</tesseract>
</truxton_options>

We want to reduce our storage by saving a file's contents only once throughout all of Truxton. This is controlled by the globaldedupe option. We would edit the TruxtonSettings.xml file and add a new element to the truxton_options section. 

<truxton_options>
  <globaldedupe>1</globaldedupe>
  <dbconnectionstring>Host=localhost;Port=5432;Database=Truxton;Username=postgres;Password=Password123;</dbconnectionstring>
  <mbconnectionstring>Host=localhost;Port=5432;Database=TruxtonMessageBus;Username=postgres;Password=Password123;</mbconnectionstring>
  <datadir>C:\Truxton Data</datadir>
  <tempdir>C:\Truxton Data\Temp\</tempdir>
  <sqlite_exploitation_scripts_folder>C:\Program Files\Truxton\Loader\SQLite Exploitation Scripts</sqlite_exploitation_scripts_folder>
  <machineid>9D6D69B5-2568-4CEC-B4D4-1A4B56FBA557</machineid>
  <solr_url>http://localhost:8983/solr/truxton-core</solr_url>
  <tesseract>C:\Program Files\Tesseract-OCR\tesseract.exe</tesseract>
</truxton_options>

Once you save the file, you must restart the Truxton service in order for the new settings to take affect.

Retrieved from "https://truxwiki.com/index.php?title=Configuration_System&oldid=10913"