Difference between revisions of "SOLR"

From truxwiki.com
Jump to navigation Jump to search
 
(4 intermediate revisions by one other user not shown)
Line 14: Line 14:
 
|}
 
|}
  
The SOLR [[ETL]] (aka SOLR Contents Indexer) indexes media contents metadata items like artifacts, messages and locations into a separately running SOLR Server. It notably does '''NOT''' index file metadata, hashes or file contents (those operations are done in [[SOLRFile|SOLRFile.exe]]).  
+
The SOLR [[ETL]] (aka SOLR Contents Indexer) indexes media contents metadata items like artifacts, messages and locations into a separately running SOLR Server. It notably does '''NOT''' index file metadata, hashes or file contents (those operations are done in [[SOLRFile]]).  
  
It occurs near the end of the load once most ETL operations are complete.
+
It occurs near the end of the load once most [[ETL]] operations are complete.
  
 
=Configuration=
 
=Configuration=
Line 31: Line 31:
 
| string
 
| string
 
| The url to the solr server and core to use
 
| The url to the solr server and core to use
|-
 
| solr_parallel_extractions
 
| number
 
| The number of extractions to do in parallel.
 
|-
 
| solr_parallel_additions
 
| string
 
| The number of addition batches to do in parallel
 
 
|}
 
|}
  
When tweaking the number of parallel extractions and addition is is recommended to not exceed the number of cores available on your exploitation machine. Additional, each parallel operation will consume additional ram in the solr instance running, so make sure you allocate enough for the JVM or you might run into out of memory issues.
+
E.G. for <code>[[Configuration_System#Truxton_Settings_File|TruxtonSettings.xml]]</code>
 
 
E.G. for TruxtonSettings.xml
 
 
<source lang="xml">
 
<source lang="xml">
 
<truxton_options>
 
<truxton_options>
Line 51: Line 41:
 
     -->
 
     -->
 
   <solr_url>http://localhost:8983/solr/truxton-core</solr_url>
 
   <solr_url>http://localhost:8983/solr/truxton-core</solr_url>
  <solr_parallel_extractions>4</solr_parallel_extractions>
 
  <solr_parallel_additions>2</solr_parallel_additions>
 
 
</truxton_options>
 
</truxton_options>
 
</source>
 
</source>

Latest revision as of 13:21, 27 July 2022

Details
Executable SOLR.exe
Stage 192
Percent Complete 90%
Message Queue solrcontentstage

The SOLR ETL (aka SOLR Contents Indexer) indexes media contents metadata items like artifacts, messages and locations into a separately running SOLR Server. It notably does NOT index file metadata, hashes or file contents (those operations are done in SOLRFile).

It occurs near the end of the load once most ETL operations are complete.

Configuration

Configurations for this ETL are nested under the root

Name Data Type Description
solr_url string The url to the solr server and core to use

E.G. for TruxtonSettings.xml

<truxton_options>
  <!-- ...
       other configs
       ...
    -->
  <solr_url>http://localhost:8983/solr/truxton-core</solr_url>
</truxton_options>