LangID

From truxwiki.com
Revision as of 11:08, 22 June 2021 by Pmuessig (talk | contribs) (Created page with "{| style="float:right;border:1px solid black" |+ Details | Executable | <code>LangID.exe</code> |- | Stage | style="text-align:center;" | 18 |- | Percent Complete | style="tex...")
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search
Details
Executable LangID.exe
Stage 18
Percent Complete 100%
Message Queue langid

The LangID ETL scans ascii and unicode files and identifies non-english language. If it is over a certain confidence threshold it will then tag the document with the language it found.

File Types

LangID processes the following types of files.

File Type Produces
Type_ASCII_Text Tags
Type_UTF8_Encoded_Text Tags
Type_Extracted_ASCII Tags
Type_Extracted_Unicode Tags

Note: The types Type_Extracted_ASCII and Type_Extracted_Unicode are typically provided by the TextExtract ETL.

Retrieved from "https://truxwiki.com/index.php?title=LangID&oldid=5064"