dtSearch Release Notes

dtSearch 7.55 (Build 7700) Released October 11, 2008

Fixes and minor enhancements

• dten600.dll: Fixed Microsoft Access parser bug causing some MDB files to be indexed very slowly
• dten600.dll: Fixed PDF file parser word breaking error affecting PDF files created using MacroMedia FlashPaper
• dten600.dll: Fixed error in MS Word 6.0 file parser affecting footnote extraction
• dten600.dll: Other file parser bug fixes (XML, MIME, Outlook MSG, Access)
• dtSearch.exe: Fixed formatting error and added diagnostic information to error message when file could not be opened
• dtSearch.exe: Added support in Edit > Copy File for copying container files other than archives

dtSearch 7.54 (Build 7680) Released September 5, 2008

Enhancements (All products)

• New Microsoft Access file parser added with no dependencies on the Microsoft JET Engine or ODBC drivers, so it works identically under Windows and Linux. This file parser supports Access databases created by Microsoft Access 95 through Access 2007.
• New file parser added for Flash (*.swf) files.

Enhancements (dtSearch Engine)

• Added support for searching for Unicode characters that are not defined as letters by default (such as currency symbols)
  By default, all Unicode characters that are defined as letters in the Unicode specification are searchable. To make other characters such as Unicode currency characters searchable, you can add a line to the end of the alphabet file listing the Unicode characters to make searchable. Example:
  

  AdditionalLetters = 00a2 00a3 00a4 00a5 20a0 20a1 20a2 20a3 20a4 20a5 20a6 20a7 20a8 20a9 20aa 20ab 20ac
  

  This example makes all of the Unicode currency characters such as the Euro, Pound, and Lira searchable characters.
• Added ListIndexFlags.dtsListIndexIncludeDocInfoAsXml flag to list all documents in an index in XML format including document properties and stored fields. The ListIndex.exe utility also has a new dx option to do this for an index from the command-line.

Enhancements (dtSearch Publish)

• Added option to enable links to Office documents (doc, xls, etc.) to launch in Office, outside of the lbview.exe program. The setting is "ExternalLaunchExtensions" in the lbview.ini file.
• See below for important information about Adobe Reader and Adobe Acrobat 9 compatibility

Adobe Reader and Adobe Acrobat 9 Compatibility

• Adobe Reader and Adobe Acrobat 9 have a new setting that disables hit highlighting in PDF files by default. To change this setting in Adobe Reader or Adobe Acrobat 9, click Edit > Preferences > Search, and check the box to "Enable search highlights from external highlight server".
• dtSearch Desktop 7.54 will automatically make this change when it detects that hit highlighting is disabled (after asking permission). dtSearch Desktop 7.54 also fixes a problem that causes PDF files to open very slowly in dtSearch with Adobe Reader or Adobe Acrobat 9 installed.
• dtSearch Publish 7.54 has new settings in the lbview.ini file to automatically enable PDF hit highlighting when it is disabled (after asking permission).
• dtSearch Web cannot change this setting in client applications, so web sites that rely on Adobe Reader to highlight hits in PDF files should notify users of the need to change this setting to preserve hit highlighting.

Fixes and minor enhancements

• dten600.dll: Outlook .msg file parser added separate table of bcc and cc recipients at end of message
• dten600.dll: Fixed error generating search reports with the "include whole file option" selected
• dtupdate.exe: Fixed error in dtupdate.exe that caused the option to install automatically after the download is complete to fail (clicking the Install button after restarting dtupdate.exe will work).
• dten600.dll: Reduced memory use indexing PDF files with very large numbers of XObject references
• dten600.dll: Fixed formatting error causing tables to appear very narrow in RTF files when converted to HTML
• dten600.dll: Reduced memory use indexing very long text files
• dten600.dll: Reduced memory use indexing very long HTML and XML files
• dten600.dll: Fixed slow PDF opening in dtSearch Desktop when Adobe Acrobat or Adobe Reader 9 is installed.
• Java API: Added SearchResults.addDoc() method to add an item to search results
• dten600.dll: Fixed incorrect formatting of some custom percent and date cell number formats in Excel files
• dten600.dll: In search reports, %%Filename%% macro in header was incorrectly replaced with PDF or HTML Title instead of the filename.
• dten600.dll: Fixed bug causing incorrect hit highlighting in search report generated from PDF file after unindexed search
• dtupdate.exe: Fixed unnecessary escalation prompt under Vista when checking for new version -- it should only prompt if there is a new version to install
• dtsfc.cpp: Fixed thread handle leak in DJobBase::startCB
• dtsearchw.exe: Fixed error in serial number validation
• dten600.dll: Fixed error processing of hyphenated words with . character defined as hyphen and "all three" hyphenation option selected

dtSearch 7.53 (Build 7629) Released May 19, 2008

Fixes and minor enhancements

• dten600.dll: Fixed error formatting combined date and time cells in Lotus 1-2-3 spreadsheet
• dten600.dll: Reduced memory use indexing XML file with very large CDATA field
• dten600.dll: Reduced memory use indexing very large HTML file
• Added IndexCache object to the Java API
• dten600.dll: Fixed incorrect page number shown in search report generated from PDF file
• Spider: Fixed timeout error crawling very slow web site
• dtWebSetup: Fixed error causing blank list under "Select search form type" in Search Controls tab of Build Search Form dialog box
• dten600.dll: Fixed error indexing PDF file with more than 32,000 pages
• dten600.dll: Fixed out of memory error caching text from very large text file
• dten600.dll: Added support for indexing attachments in PDF files
• dten600.dll: Fixed incorrect hit highlighting on PDF page containing Unicode hyphen characters when hyphens indexed as letters

dtSearch 7.52 (Build 7600) Released April 8, 2008

dtSearch Engine

• 64-bit Java API
• 64-bit version of the dtSearch Engine for Linux
• Option to index .eml files as containers (so the message body and attachments are each indexed as a separate file).
• Sample ASP.NET application implementing an OpenSearch interface to the dtSearch Engine

Fixes and minor enhancements

• dten600.dll: To improve consistency in the handling of punctuation in field names, unsearchable characters are now removed from field names in input data, with a few exceptions (:&_+=.) to minimize the effect on backward compatibility. In previous versions this was done in indivdual file parsers so the effect of punctuation in field names depended on the format of the input data.
  This change will not generally affect searching because only searchable letters are used when matching field names.
  This change may affect the field names associated with stored fields, in cases where the field name contains punctuation characters.
• dten600.dll: Fixed two bugs affecting field parsing in Office 2007 documents.
• dten600.dll: Fixed incorrect document count displayed in history.ix when startingDocId set to value other than 1
• dten600.dll: Fixed XML hit highlighting bug resulting in illegal   entity appearing in XML output
• dten600.dll: Fixed bug causing extra "F" character to appear in stored fields
• dten600.dll: Improved handling of low-memory conditions for searches that generate very large search results sets
• dten600.dll: Fixed error caching text in index when an external language analyzer returns overlapping blocks of text
• Added: FileConvert.exe and ListIndex.exe command-line utilities utilities
• dtSearch.Spider2.dll: Added LinkTraceFilename option to create a log of the links followed and not followed during a crawl
• Added dtsLaJob.searchRequestPunct string advising external language analyzers of the search request punctuation characters to preserve when analyzing a search request
• dtSearchNetApi2.dll: If data source throws an exception, IndexJob will catch it and report the exception through the Errors object
• dtSearchNetApi2.dll: Added FileInfoFlags.fiOpenFailed to indicate when a document returned from a DataSource with the DocIsFile flag set to true cannot be opened because it is either not present or locked
• dten600.dll: Fixed merge bug affecting merges of indexes containing the same container file with the newer version of the container file in the target index

dtSearch 7.51 (Build 7556) Released January 18, 2008

dtSearch Desktop

• Search results saved as XML include the selection state of the items in the search results list (i.e., checked or unchecked). Search results saved in other formats such as CSV can either include all items or just selected items.

dtSearch Web

• Added 64-bit version of dtSearch Web and dtSearch Web Setup

Fixes and minor enhancements

• dtSearch.Spider2.dll: Added 64-bit version of the .NET Spider API
• dtSearchw.exe: Added status bar indicator for total hits and total number of files retrieved
• lbvprot.dll: Fixed error in standard CD type causing it to get stuck at an "Opening page..." message
• dten600.dll: Fixed file parser error causing formatting errors and extraneous text in MS Works documents
• dten600.dll: File parsers added for Lotus 123 and Quattro Pro
• dten600.dll: Fixed incorrect highlighting in search report generated from cached text when automatic CJK word breaking enabled
• dten600.dll: Fixed bug in HTML file parser that could cause duplication of field values in stored fields extracted from HTML meta tags
• dten600.dll: Fixed HTML hit highlighting bug that caused comment tags to appear inside the <TITLE>

dtSearch 7.50b (Build 7518) Released November 25, 2007

Fixes and minor enhancements

• dten600.dll: Fixed SearchReportJob error causing more blocks of context than specified by MaxContextBlocks to be included in the generated report.
• lbvprot.dll: Fixed error starting CGI applications

dtSearch 7.50 (Build 7517) Released November 9, 2007

Enhancements (All products)

• Improved integration with external language analyzers: (1) Language analyzers will be given much larger chunks of text to analyze, which enables some language analyzers to operate more effectively. (2) Language analyzers will be given consistently-sized chunks of text whether indexing or highlighting hits, which ensures that hit highlighting will not be affected by changes in the behavior of a language analyzer depending on the size of the data it receives.

dtSearch Desktop

• In the Edit > Copy File dialog box, added option to preserve original modification, creation, and last access times of the original files
• In the Edit > Copy File dialog box, added option to copy the entire container file when a matching document is inside a container (such as a ZIP file or email archive)
• Option in Options > Preferences > Indexing Options > Letters and Words to automatically insert a word break around Chinese, Japanese, and Korean characters in text. This makes it possible for documents that do not contain word breaks to be searched.

dtSearch Engine

• 64-bit version of the dtSearch Engine with C++ and .NET APIs.
• New dtsoTfAutoBreakCJK flag in Options.TextFlags to automatically insert a word break around Chinese, Japanese, and Korean characters in text. This makes it possible for documents that do not contain word breaks to be searched.
• ListIndexJob added to the Java API
• New dtsListIndexIncludeDocCount flag added to ListIndexFlags, to provide a the document count for each word listed

Fixes and minor enhancements

• dten600.dll: Added dtsLaJob.pFileInfo to provide language analyzer with a dtsFileInfo describing the document being processed
• dten600.dll: Added dtsLaJobInputIsFirstBlockInDocument value for dtsLaJob.flags to tell language analyzer when a new document is starting
• dten600.dll: Fixed error formatting generated table of contents in .docx file
• dtIndexerw.exe: MAPI_E_CALL_FAILED error indexing Outlook data with Outlook 2007
• dten600.dll: Fixed crash indexing Word document with corrupt styles
• dten600.dll: Fixed PDF parsing error causing incorrect word break
• dten600.dll: Added support for non-Microsoft variant of Microsoft Searchable TIFF format
• dten600.dll: Fixed error generating search report when SearchReportJob specified neither an OutputFile nor OutputToString (fixed in build 7517)

dtSearch 7.43 (Build 7476) Released September 16, 2007

Fixes and minor enhancements

• dten600.dll: Fixed error processing list formatting in some MS Word files caused bullets to be rendered as numbers.
• dten600.dll: Fixed errors formatting text in .docx files.
• dten600.dll: Fixed bug in PDF file parser affecting decoding of CID fonts in PDF files
• dten600.dll: Fixed error extracting item from TAR file to hit-highlight after search
• dten600.dll: Added detection of the following file types with missing or incorrect filename extensions: Microsoft Word 2003 XML files, Microsoft Excel 2003 XML files.
• dtsjava.dll: Fixed error indexing using data source API under WebSphere
• dten600.dll: Fixed extra spacing in output when HTML converted to UTF-8 text

dtSearch 7.42 (Build 7467) Released July 31, 2007

Enhancements (All products)

• Added support for Microsoft Searchable TIFF (created by Microsoft Office Imaging), Microsoft Document Imaging (*.mdi), Windows Metafile *.wmf) and Enhanced Metafile (*.emf) formats

Enhancements (dtSearch Engine)

• Added flags to control recognition of ambiguous dates (new TextFlags values dtsoTfRecognizeDatesPresumeDMY, dtsoTfRecognizeDatesPresumeYMD)

Fixes and minor enhancements

• dtSearchNetApi2.dll: JobErrorInfo no longer requires or supports the IDisposable interface.
• dten600.dll: Filenames longer than 1024 characters could cause "duplicate filename" errors when verifying an index.
• dtSearchw.exe: Fixed problem with mouse wheel scrolling in search results window with high-resolution mouse wheels
• dten600.dll: Improved recognition of CJK encodings in HTML and PDF
• dten600.dll: Fixed error in title attribute for documents indexed using the COM implementation of the data source API
• dten600.dll: Changed the behavior when a container document such as a ZIP file is removed from an index using its doc id (using IndexJob.ActionRemoveListed). Instead of just removing the container, the container and all contained items will be removed.
• dten600.dll: MS Word file parser displayed some internal field data (TC, TA, SEQ)
• dten600.dll: Added detection of .docx, .xlsx, .xps, and OpenOffice documents with missing or incorrect filename extensions

dtSearch 7.41 (Build 7420) Released April 21, 2007

Enhancements (dtSearch Engine)

• Added support for automatically varying hit weights according to the field they occur in, through the new SearchJob.FieldWeights setting. For more information, see the "Relevance" topic in the dtSearch Engine API Reference.
• Added improved progress reporting during unindexed searches to the C++ API (see dtsSearchProgressInfo in the dtSearch Engine API Reference) and the .NET 2.0 API (see ISearchStatusHandler2 in the .NET 2.0 API Reference).
• Compatibility note for developers working with the .NET 2.0 API only:  The DLL dependencies for dtSearchNetApi2.dll have changed due to Visual Studio .NET 2005 Service Pack 1.   Because dtSearchNetApi2.dll is built with Service Pack 1, it requires the updated MFC and CRT DLLs that are included with that version.  Executing the vcredist_x86.exe included with Visual Studio .NET 2005 Service Pack 1 (dated December 2, 2006 or later) will install these components. 
  This issue does not affect any other dtSearch Engine API.

Fixes and minor enhancements

• dten600.dll: New fields added as properties of .eml files - CC, BCC, and Attachments (a list of the filenames of all attachments).
• dtSearch .exe and .msi files digitally signed for better operation in Windows Vista
• dtSearchNetApi2.dll: Error in ConvertPath caused unnecessary refresh of virtual path mappings from the metabase.
• dten600.dll: Minor improvements in the binary file detection and Unicode filtering algorithm for binary files
• dten600.dll: Fixed bug in MHT file parser that caused hit highlighter to generate blank HTML page for some MHT files
• dten600.dll: IndexCache object added to the COM interface.
• dtSearchw.exe: Fixed error starting indexer and dtSearch Web Setup under Windows Vista

dtSearch 7.40 (Build 7360) Released February 22, 2007

Enhancements (All products)

• Added automatic recognition of dates, email addresses, and credit card numbers in text. For more information, see http://www.dtsearch.com/dateRecog.html
• Added support for Vista (XMP) metadata in .jpg and .tif images.
• Added support for PowerPoint 2007 (*.pptx).
• Added support for Vista XML Paper Specification (*.xps) documents.

Enhancements (dtSearch Engine)

• Added IndexCache object in the .NET 2.0 API, and dtsIndexCache object in the C++ API, to enable much faster searching when a series of searches must be done against a small number of indexes. The IndexCache maintains a thread-safe pool of open indexes that are available for searching during the lifetime of the cache. Using the cache eliminates the need to open and close the index for each search

Enhancements (dtSearch Desktop)

• Added option in Options > Preferences > Spider Options to log the links found in each page the Spider follows.
• Added option in Options > Preferences > Search Options to change the maximum number of words a search request can match.

Fixes and minor enhancements

• dtSearch.exe: Fixed "Invalid Character" error displaying documents in report view after installing Internet Explorer 7.
• dten600.dll: When serializing stored fields to XML, add a _ in front of any stored field names that begin with a digit so the resulting XML remains syntactically correct.
• dten600.dll: In the C++ API, the pOnIndexWordFn callback was called with encoded field information in addition to the text of the word, and if the called function did not preserve this field information intact, field attributes could become invalid. To prevent this, in version 7.40 the field information is removed before the callback so pOnIndexWordFn will not see or be able to affect field attributes.
• dten600.dll: Increased the maximum value for MaxWordsToRetrieve to 512k (from 256k)
• dten600.dll: When checking for available disk space in a folder, indexer did not check whether the folder was mounted from a different physical drive.
• dtSearch Web: Added option to log BooleanConditions and FileConditions to search log.
• dtindexerw.exe: Fixed MAPI_E_UNKNOWN_FLAGS error indexing Outlook messages in Outlook 2002 (fixed in build 7360)

dtSearch 7.30 (Build 7320) Released September 30, 2006

Enhancements (All products)

• Added preliminary support for Word 2007 (*.docx) and Excel 2007 (*.xlsx) based on the current Office 2007 beta and available documentation.
• Added support for JPG and TIFF metadata, including EXIF and IPTC fields.
• Unicode filtering file parser can handle individual documents larger than 2 Gb, and support for files larger than 2 Gb added to the extext.exe utility
• Improved handling of partially inaccessible email files. In previous versions, if an email had encrypted or corrupt data (for example, an encrypted attachment), the whole email was reported as encrypted or corrupt. In this version, the readable portion of the message is indexed and the unreadable portion is separately reported as a partially encrypted or partially unreadable file. This change applies to Outlook messages, TNEF files, .eml files, MBOX archives, and .msg files.

Enhancements (dtSearch Engine)

• Beta x64 (64-bit) versions of the dtSearch Indexer and dtSearch Engine (dtIndexer64.exe, dtengine64.dll, and dtSearchNetApi2.dll. The index format and APIs (C++, COM, and .NET) are identical to the 32-bit version. The 64-bit components are in a separate download file (dtSearch64_730.exe) with the same installation password as the dtSearch Engine SDK.
• Added alternative PDF highlighting mechanism for client-based applications (see "Highlighting Hits in PDF files" in the API Overviews section for details)
• Added ListIndexJob object to the .NET 2.0 API to list files, words, or fields in an index (see dtSearchNetApi2.chm for API reference)
• Added dtsListIndexIncludeDocId flag for dtsListIndexJob and ListIndexJob to provide a quick way to list all documents in an index and the doc id for each document
• C++ API Changes to support 64-bit file sizes in dtsInputStream (added size64 and seek64), dtsInputStreamReader, dtsFileInfo (added size64), dtsSearchResultsItem (added size64). These changes preserve binary compatibility for the dtSearch Engine DLL, but some C++ code may trigger new warnings when compiled because of 64-bit values returned.
• Added dtsIndexKeepExistingDocIds flag to specify that, when compressing an index, the indexer should not remap document ids, so document ids will be unmodified in the index once compression is done.

Fixes and minor enhancements

• dtWebSetup.exe: Fixed bug causing "Build Search Form" tool to create extra button bar when overwriting an existing form
• dten600.dll: Fixed 'out of memory' error verifying very large index
• lbviewer.exe: Fixed bug causing XML to appear incorrectly when displayed using a stylesheet
• dten600.dll: PowerPoint file parser - added support for embedded OLE objects
• dten600.dll: PDF file parser detects and handles case where text in right-to-left languages (Hebrew or Arabic) is stored backwards (left-to-right) in a PDF file, and automatically inverts the characters in the word so it will be correctly searchable
• dten600.dll: PDF file parser handles invalid PDF files created by OCR product that leaves out required /Pages and /Page tags in PDF structure
• dtSearchNetApi2.dll: JobErrorInfo object did not implement IDisposable interface, preventing deterministic release of allocated resources.

dtSearch 7.25 (Build 7285) Released June 25, 2006

Fixes and minor enhancements

• dtWebSetup.exe: Fixed bug causing dtSearch Web Setup to fail to run on some Windows 2003 Server systems
• dten600.dll: Added Fragmentation, ObsoleteCount, and IndexFlags to the COM IndexJob.GetIndexInfo() and Java IndexJob.getIndexInfo() methods. Also, indexing dates are now reported as a date and time.

dtSearch 7.24 (Build 7245) Released June 7, 2006

Enhancements (dtSearch Engine)

• Added support for indexing and searching TNEF (Transport Neutral Encapsulation Format) files
• dtSearch.Spider .NET API has new Authentication, FormAuthentication, and ProxyInfo properties

Fixes and minor enhancements

• dten600.dll: EML parser - fixed bug indexing messages with no message body
• dten600.dll: Excel file parsing bug caused indexer to hang on corrupt Excel file
• mapitool.exe: Added workaround for MAPI error MAPI_E_NOT_ENOUGH_MEMORY when saving a message that has a very large number of recipients (see http://support.microsoft.com/kb/171907).
• dten600.dll: RTF parser - fixed bug parsing headers in RTF files
• dten600.dll: Improved parsing of Ole10Native streams in OLE Storage files
• C++ Source code samples reorganized into vc6, vc7, and vc8 folders
• dten600.dll: Fixed PDF hit-highlighting bug that caused highlighting to fail to appear in some documents

dtSearch 7.23 (Build 7241) Released May 8, 2006

Enhancements (dtSearch Desktop)

• Added new Indexing Resources preferences page
• Added pause button to Update Index dialog box

Enhancements (dtSearch Engine)

• Added ASP.NET 2.0 sample applications in VB.NET and C# in C:\Program Files\dtSearch Developer\examples\asp.net2
• Java API: empty() method added to SearchResults, and clear() method added to SearchReportJob and SearchJob to force deterministic release of memory allocated for SearchResults

Fixes

• dten600.dll: PowerPoint file parsing bug caused incorrect character formatting
• dten600.dll: Word file parsing bug caused right-to-left text to be left-aligned instead of right-aligned by default.
• dten600.dll: EML parser - improved detection and parsing of malformed .eml files.
• mapitool.exe: Fixed error converting UTC date when saving .msg files to disk
• dten600.dll: Fixed crash indexing corrupt .mp3 file
• libdten600.so (Linux): Fixed error parsing zipped files with accented characters in the filename
• dtsearch.exe: Fixed error detecting Ctrl+C keyboard shortcut
• dten600.dll: Excel file parser bug caused some valid XLS files to be incorrectly reported as corrupt

dtSearch 7.22 (Build 7217) Released March 14, 2006

Enhancements (dtSearch Engine)

• .NET 2.0 API for Visual Studio .NET 2005. The .NET 2.0 API wrapper is dtSearchNetApi2.dll, and the .NET 2.0 version of the Spider API is dtSearch.Spider2.dll. The API is identical to the .NET 1.1 API. For sample code, see the examples\cs2 and examples\vb.net2 folders.
• Added dtsSortByFullName sort flag
• Added docByteRead64, bytesRead64, bytesToIndex64 to dtsIndexProgressInfo
• Added Options.StoredFieldDelimiterChar, which provides a way to specify a delimiter between multiple instances of a stored field in a single document

Enhancements (dtSearch Desktop)

• Added option to dtSearch to view PDF files as plain text instead of using Adobe Reader (Options > Preferences > External Viewers).

Fixes

• dten600.dll: Fixed indexer bug that caused index to be reported as having "Illegal ref ptr" error when verified
• dten600.dll: Fixed Excel file parser bugs affecting formatting of date and time values
• dten600.dll: Fixed indexing crash when indexing very large .gz files
• dten600.dll: Minor improvements to word break detection in PDF files
• dten600.dll: Fixed RTF file parser bug that could cause indexing crash on corrupt RTF file
• dten600.dll: Excel file parser defaults to 10 digits of precision for numbers without a specified format (consistent with Excel).
• dten600.dll: Minor improvements to Unicode filtering algorithm.

dtSearch 7.21 (Build 7164) Released January 23, 2006

Enhancements (All products)

• IFilter support to enable dtSearch to parse document types such as Microsoft OneNote and AutoCAD that include IFilters.   
  IFilters are components that enable various Microsoft search products, such as Microsoft Index Server, to extract text from documents. For example, when you install Microsoft OneNote, an IFilter is installed to enable searching of *.one files. To tell dtSearch to use installed IFilters to process some of your files, set up a rule in Options > Preferences > File Types and under File type, select "IFilter". In dtSearch Engine applications, use the FileTypeTableFile to specify the filename patterns to use with IFilters. The IFilter adapter only works on systems with the Microsoft component query.dll installed.
  For information on products that include query.dll, see http://support.microsoft.com/dllhelp
  For more information on IFilters, see http://www.ifilter.org/ or http://channel9.msdn.com/wiki/default.aspx/Channel9.DesktopSearchIFilters

Fixes

• dten600.dll: Fixed bug that prevented some items in ZIP files from being displayed after a search (an "unable to access input file" message would appear instead).
• dten600.dll: Fixed bug in file parsers for Microsoft Office documents (PowerPoint, Excel, and Word) that could cause dtSearch to crash attempting to index corrupt documents
• dten600.dll: Fixed bug in PDF file parser that caused "Bad xref" error on some PDF files created with PDF 1.5-only compatibility
• dtsearch.h: unnamed unions removed from the dtsMessage structure. This will not affect binary compatibility but may require source code changes in C++ code that accessed undocumented union members. Because the removed union members were undocumented, this change should affect very few programs.

dtSearch 7.20 (Build 7136) Released December 6, 2005

Enhancements (All products)

• New file parsers for OpenOffice documents, spreadsheets, and presentations (*.sxw, *.sxc, *.odt, *.ods, etc.), covering OpenOffice version 1 and OpenOffice version 2 (the "Open Document Format for Office Applications")
• New file parsers for the Microsoft Office XML formats (Microsoft Word 2003 XML and Microsoft Excel 2003 XML)

Enhancements (dtSearch Desktop)

• Added "Opening containing folder" in right-click menu for retrieved items
• Improved reporting of errors that occur when copying files in Edit > Copy File(s)
• dtindexer.exe: added /caf and /cat command-line option to cache text (/cat) or cache original files (/cad), when creating indexes using the command line, and /recog to recognize an index.
• Added Help > Check For Updates feature to automatically download new versions

Enhancements (dtSearch Engine)

• dtSearch.Spider.dll component provides a .NET API for the dtSearch Spider. For API documentation, see dtSearchNetApi.chm. For sample code, see C:\Program Files\dtSearch Developer\examples\cs\SpiderDemo.
• New xfilter search type, "ext", to search only on the filename extension (dot required). Examples: xfilter(ext ".doc") matches file with a .doc extension; xfilter(ext "~.doc") matches file without a .doc extension; xfilter(ext ".") matches file with no extension. This search feature will only work with documents that were indexed with dtSearch 7.2 or later.
• SearchReport supports %%FirstHit%% macro in ContextHeader to indicate the word offset of the first hit in the context block
• dtsIndexCacheTextWithoutFields flag added to IndexingFlags. This flag makes it possible to cache text (for generation of a synopsis to include in search results) without including any of the fields added using the data source API.
• dtsErAccCachedDoc flag added to ErrorCodes. This error code indicates that a document could not be extracted from the document cache in an index (this usually means that the index was created without caching enabled)
• dtsConvertJustDetectType flag added to ConvertFlags, to have FileConverter or DFileConvertJob just detect the file format of a document. The format is returned in FileConverter.DetectedTypeId.
• dtSearch Engine for Linux updated to the dtSearch 7.2 code base; multithreading support added
• dtsReportIncludeFileStart flag added to ReportFlags. This flag causes a block of text from the beginning of the document to be included in the generated search report.
• A new search feature makes it possible to restrict a search to the text of documents (excluding any metadata). To search for text that is not in any field, search for //text contains (search request). Example: (//text contains apple) and (author contains smith)

Fixes

• DynaZip unzip component (dunzip32.dll) updated to new version that eliminates buffer overrun vulnerability in earlier versions.
• dtSearch.exe: Installing Adobe Acrobat 7.05 update caused hit highlighting to stop working in PDF files
• dten600.dll: Reduced amount of memory needed to parse very large Word, Excel, and Outlook items
• dten600.dll: Fixed file parsing error in Word documents that caused bullets to be rendered as auto-numbered lists
• dten600.dll: Changed handling of CSV files that do not have a header that lists field names; these files are now handled as plain text, since no field information is available for them.
• Spider: Fixed: Links with // in the name (http://www.example.com//default.html) caused index to be reported as corrupt by Verify Index
• dtSearch.exe: MAPI profile id, entry id, and store id were displayed in search results list
• dtSearch Web: WebSearchForm.js incorrectly handled blank filename filter
• Spider:  Did not match port number against filename filters for port numbers other than 80
• dtSearch.exe:  Displayed MAPI entry id and store id of Outlook messages in search results.

dtSearch 7.10 (Build 7045) Released August 8, 2005

Enhancements (dtSearch Engine)

• Added two new ASP.NET samples, one in VB.NET and one in C#, that demonstrate a search interface using a grid control for search results. The new samples are installed to C:\Program Files\dtSearch Developer\examples\asp.net. Please see the readme file in the project folders before trying to open them in Visual Studio -- a virtual directory mapping for C:\Program Files\dtSearch Developer\examples\asp.net has to be created first or Visual Studio will not be able to open the project.
• GetNthWordDocCount added to WordListBuilder to get the number of documents a word occurs in
• SearchReportJob enhancements: Added ContextSeparator; itUnformattedHTML output format, for easier generation of a synopsis; faster generation of search report when search results cover multiple indexes; dtsReportLimitContiguousContext flag to prevent very large synopsis when there are many hits close together.
• In the OnFound callback notification in the C++ and .NET interfaces, an application can veto individual items to prevent them from being included in search results. See SearchResultsItem.VetoThisItem (.NET) and DSearchJob::VetoThisItem (C++ Support Classes).
• dtSearchNetApi.dll uses registry type library information and delay loading to eliminate the need for dten600.dll to reside on the system PATH in ASP.NET applications.
• New TextFlags option to suppress automatic generation of xfirstword and xlastword (dtsoTfSkipXFirstAndLast)
• Options.MaxFieldNesting setting to limit the permissible depth of field nesting
• .NET API objects implement Dispose() for more deterministic release of allocated resources.
• .NET IndexJob.ExecuteInThread and SearchJob.ExecuteInThread use .NET thread pool instead of creating a thread.
• dtSearch Engine for Linux updated to the dtSearch 7.1 code base

Enhancements (dtSearch Publish)

• "Standard" CDs (which use lbview.exe) can launch non-CGI programs from the CGI-BIN folder using a new URL syntax. See the "Standard CDs" help topic in dtSearch_Web.chm for details.
• "Standard" CDs can highlight hits in PDF files with Adobe Reader 6 or later (formerly Adobe Reader 7 was required).

Fixes

• dten600.dll: Reduced amount of stack required to process very long xfilter expressions
• dten600.dll: Fixed index merge bug that could cause a corrupt index merging into a large index without the "clear target" flag set
• dten600.dll: Bug caused MakePdfWebHighlightFile to return a blank string after unindexed search
• dten600.dll: The default value of Options.MatchDigitChar has changed from blank (disabled) to '=', to be consistent with the behavior of dtSearch Desktop.
• dten600.dll: Bug caused unindexed search of HTML field defined using comment tags to find the same search term outside of the field

dtSearch 7.01 (Build 7025) Released June 14, 2005

Enhancements (dtSearch Web)

• Generated search form has more flexible stylesheet references
• File parsers generate HTML output using "em" units for font sizes instead of points, which allows font sizes to scale up or down in Internet Explorer

Enhancements (dtSearch Publish)

• Added "Recognize CD" function to use the CD Wizard to modify a CD that was created on a different computer

Fixes

• dten600.dll: MS Word file parser caused a word break when MS Word inserted redundant font changes within a word
• lbview.exe: Error opening PDF file with URL-encoded apostrophe in filename or path
• dten600.dll: PowerPoint file parser error parsing slide without outline entry text
• dtSearchNetApi.dll: SearchResultsItem did not include modified date or type id
• dten600.dll: SearchResults did not read HitsByWord when serializing from XML
• dtSearch Publish: PDF files did not highlight hits in Adobe Reader 7 in some systems with unpatched versions of IE components.
• dtIndexer.exe:  Default setting for IndexAutoCommitIntervalMB forced large index updates to commit too frequently, making indexing slower.

dtSearch 7.00 (Build 7008) Released May 18, 2005

Enhancements (All products)

• High-capacity index format released, with support for over 1 terabyte of data per index.
  dtSearch 7 can update and search indexes created with dtSearch 6.
  To upgrade an index to the version 7 format using dtSearch Desktop, (1) click Index > Update Index..., (2) Check the box to "Upgrade index to version 7 format". (3) Click "Start Indexing"
• New variable field weighting search option. Example: "(Description:5 contains (apple and pear)) or (author:2 contains smith)"

Enhancements (dtSearch Desktop)

• Added Spider option to pause between page downloads, to reduce them impact on the server of a crawl
• PDF files open faster in Adobe Reader if Adobe Reader 7 is installed

Enhancements (dtSearch Engine)

• New API documentation. See dtSearchApiRef.chm (overviews, C++, and COM interface), dtSearchJavaApi.chm (Java interface), and dtSearchNetApi.chm (.NET interface)

Enhancements (dtSearch Publish)

• A new file-based CD interface has been added that does not rely on HTTP.
• The CD Wizard has been simplified

Fixes

• dtSearchNetApi.dll: repeated instances of first element returned in HitsByWord array
• dten600.dll: Hidden stored fields (fields with names prefixed by **) were stored with the ** mark in front of the field name, causing serialized XML search results to have incorrect XML syntax
• dten600.dll: Check for available disk space did not handle volume sizes larger than 2 Terabytes
• mapitool.exe:  several minor bug fixes
• dten600.dll: MS Word file parser did not handle FORMTEXT fields
• dtSearch.exe: fixed 800a025e error from Internet Explorer when dtSearch tries to select hidden text to highlight from a Word document
• dten600.dll: Fixed PDF file parser error counting words in pages with annotations, causing incorrect highlighting
• dten600.dll: Fixed hit highlighting error in MIME files
• dtv_odbc.dll: Fixed bug handling Unicode data in Access fields
• dten600.dll: Added current option settings in effect to history.ix entry for each update
• dtisapi6.dll: Urls not truncated using MaxUrlSize

dtSearch 6.5 (Build 6608) Released January 18, 2005

Enhancements (All products)

• Improved file parsers for Microsoft Office file formats (Word, Excel, PowerPoint, Outlook MSG).
• New filtering option to apply text filtering to automatically recover text from corrupt documents
• Beta support for new high-capacity index format.

Enhancements (dtSearch Desktop)

• Faster indexing of Outlook message stores. Indexing speed is substantially faster, especially for incremental updates. For compatibility, existing Outlook indexes will continue to use the previous indexer, so the improvements will only apply to new indexes.
• Edit > Copy File can copy retrieved Outlook messages as .msg files (formerly they were converted to HTML). This only works for Outlook messages indexed using the new Outlook indexer (see above).
• mapitool.exe command-line utility to convert PST files or other Outlook-accessible message stores to MSG files. See mapitool.html for documentation.

Enhancements (dtSearch Engine)

• Support for hidden stored fields. Hidden stored fields are returned in search results like stored fields, but are not displayed as part of the document and are not searchable. Only fields returned through the data source API in DocFields (.NET/COM) or dtsInputStream.fields (C++) can be hidden stored fields. To designate a field as hidden, insert ** in front of the field name.
• Note: dtv_ms.dll, an external file parser included in prior versions, is now compiled into dten600.dll.

dtSearch 6.4 (Build 6482) Released September 4, 2004

Enhancements (All products)

• Improved indexing speed
• Support for indexing .tar, .gz, and .tgz archives
• Support for indexing metadata (Author, Title, etc.) in MP3, ASF, and WMV files
• Improved file parser for RTF files.
• XML parser improvements in handling incorrect XML input, such as mismatched tags and > and < characters in field text

Enhancements (dtSearch Web)

• Web search form remembers users' search criteria from prior searches
• Many improvements to search form generation: more controls including date range control, simpler search forms, automatic generation of field searching controls; stylesheet-based search form and search results formatting; Form Builder remembers search form option settings from previously-built form
• dtSearch Web Setup has new options to upgrade all dtSearch Web installations at once, and to remove a dtSearch Web installation
• dtSearch Web option to encrypt index paths in search forms

Enhancements (dtSearch Desktop)

• Added support for forms-based authentication in dtSearch Spider
• Improved traversal of JavaScript-generated links in dtSearch Spider
• Option to suppress indexing of HTML tags in XML data

Enhancements (dtSearch Engine)

• Native .NET Interface. For API documentation, see help\dtSearchNetApi.chm.
• Added to COM interface: Verify and Merge indexes, and IndexJob.IsThreadDoneWait.
• Added to Java interface: Verify and Merge indexes.
• Added dtsIndexProgress.fullname
• Added new ASP sample application demonstrating display of synopsis in search results
• LZW decompression of PDF files is now always enabled, since the Unisys patent has expired

Fixes

• Error in decryption of encrypted PDF files with SecurityModel = 2
• dtsearchw.exe: Error opening PDF file on systems with network configuration errors
• dtisapi6.dll: Error sorting search results by user-defined field when combined with with numerical flags
• dtisapi6.dll: Fixed bug causing incorrect XML to be returned from query
• dtSearchw.exe: PDF highlighting URL used localhost instead of 127.0.0.1, causing unnecessary DNS access
• dten600.dll: Fixed hyphenation bug that could cause an index to be reported as corrupt when documents are indexed with the dtsoHyphenAll hyphenation option
• dten600.dll: Fixed two bugs in .msg file parser
• dtindexer.exe: Fixed blank index name and path in "index created" message when using "Create Index - Advanced"
• dten600.dll: Fixed error indexing MBOX archive containing corrupt message data

dtSearch 6.33 (Build 6430) Released April 21, 2004

Fixes

• Outlook indexing stops working after installing Microsoft Office XP Service Pack 3
• dtSearch Web bug fixes

dtSearch 6.32 (Build 6429) Released March 5, 2004

Enhancements (All products)

• Improved relevancy ranking using positional scoring, which ranks documents higher when hits occur near the top of the file or are clustered within a document. In dtSearch Desktop and dtSearch Web, positional scoring is applied automatically when automatic term weighting is selected. In dtSearch Engine applications, use the new dtsSearchPositionalScoring search flag to enable positional scoring.

Enhancements (dtSearch Spider)

• Added option to allow Spider to crawl across multiple servers from a single starting URL
• Added option to limit maximum size of items Spider can download from a site
• Added option to limit number of files Spider can index on a web site
• Added option to limit number of minutes Spider can spend indexing a single web site
• Spider supports the "robots" meta tag

Enhancements (dtSearch Engine)

• Added SearchResults.setSortKey to Java API
• Large search results sets stored more efficiently, substantially reducing memory use
• Added TextFlags option dtsoTfSkipNumericValues. By default, dtSearch indexes numbers both as text and as numeric values, which is necessary for numeric range searching. Use this flag to suppress indexing of numeric values in applications that do not require numeric range searching. This setting can reduce the size of the index by about 20%.
• Added dtsoFfSkipFilenameFieldPath FieldFlag, to allow indexing of the filename as a field without the whole path

Enhancements (dtSearch Publish)

• Added option to specify the browser to launch
• Added options to specify format of folder listings

Fixes

• dten600.dll: Error processing user-defined synonym containing only punctuation
• dtindexer.exe: In some situations, Spider would not immediately stop downloading a long file when cancel button pressed during indexing
• dten600.dll: Enhanced error detection in index merge, and merge events logged to history.ix file
• dten600.dll: Merge Indexes bug could cause merge job to terminate with incorrect "index corrupt" message
• dtsearchw.exe: Unnecessary dependency on secur32.dll on some Windows NT 4.0 systems

dtSearch 6.31 (Build 6393) Released November 24, 2003

Enhancements (All products)

• Added automatic detection of MBOX-format email archives

Fixes

• dtIndexer: The Windows TEMP folder was used instead of the Spider temporary folders setting to determine the location of temporary files during indexing of a web site
• dten600.dll: Fixed bug in index merge function that could cause the target index to be reported as corrupt after a merge
• dten600.dll: Fixed word breaking error in PDF file parser
• dten600.dll: Fixed Spider error handling URL containing &amp HTML entity
• dten600.dll: Fixed PDF parsing error that caused trademark symbol ™ to appear incorrectly in document properties fields

dtSearch 6.30 (Build 6386) Released November 11, 2003

Enhancements (All products)

• Added pre/N connector, which is like W/N but requires that the first expression occur before the second

Enhancements (dtSearch Desktop)

• Added option in Options > Preferences > Filtering Options to control the minimum size of a text segment
• Added option in Options > Preferences > Search Results to remember the sort order from the previous search. For example, if you click the Filename column to sort search results by filename, after the next search results will be displayed sorted by filename. The remembered sort order overrides whatever was selected in the Search dialog box.

Enhancements (dtSearch Web / dtSearch Publish)

• dtSearch Web search speed has been increased substantially.
• dtSearch Publish CD access program automatically detects when the user's browser window closes

Enhancements (dtSearch Engine)

• Language Analyzer API for integration of third-party language tools (such as a Japanese or Chinese dictionar-based word breaker, or an Arabic morphological analyzer). For more information, see the "Language Analyzer API" topic in the dtSearch Engine help file, dtengine.chm

Fixes

• dten600.dll: Text Fields limited to first N lines were applied through entire file for WordPerfect files
• dtSearch: Fixed bug affecting internal links in HTML files opened after a search
• dtIndexer: Bug in index update scheduler prevented a scheduled task from being created if another task had previously been created, and deleted, with the same name
• dten600.dll: Search report generated using exact words of context contained an incorrect character at end of a context block if the last word in the context block was followed by a Unicode punctuation mark.
• dtSearch: After installing Adobe Reader or Acrobat 6, PDF files opened very slowly in dtSearch Desktop.
• dten600.dll: In MIME-encoded emails with multipart/alternative encoding (message included in both HTML and text), message text appeared twice.
• dten600.dll: Error indexing files with names longer than 260 characters

dtSearch 6.21 (Build 6345) Released August 25, 2003

Enhancements (All products)

• Support for indexing Treepad HJT files.
• Support for indexing NTFS document summary information fields
• Improved indexing of Outlook .MSG files

Enhancements (dtSearch Desktop)

• Added option to automatically open Adobe Reader in the background before opening PDF files.  (This makes PDF files open more quickly in dtSearch, especially with Adobe Reader 6.)

Enhancements (dtSearch Engine)

• A SearchFilter can be constructed from the results of a SearchJob. See SearchJob.WantResultsAsFilter in the dtSearch Engine help file for more information.
• Documents can be removed from an index by DocId rather than by filename. See "Building and Maintaining Indexes" in the dtSearch Engine help file for more information.
• dtsoFfShowNtfsProperties flag (in Options.FieldFlags) to enable indexing of NTFS document summary information fields.
• dtsConvertXmlToXml flag enables FileConverter to highlight hits in XML documents with XML output. For more information, see "Highlighting hits in XML" in the dtSearch Engine help file, tech support article dts0183, and this demo page.

Fixes

• dtSearch: TAB key to switch between search results and document window did not work
• dtIndexer: The "Update Multiple Indexes" dialog box could add web site content from an index to subsequently-updated indexes in the same session
• dtSearch: Fixed: Edit > Copy File with "Preserve folder names" did not correctly handle Outlook message with / in the message subject
• dten600.dll: Fixed bug in index merge function that could cause the target index to be reported as corrupt after a merge
• dten600.dll: ContextHeader omitted in first block of context in SearchReport, if report generated using paragraphs of context
• dten600.dll: Duplicate items in index when a list of files to index is passed to the indexer with duplicate items in the list of files
• dten600.dll: Fixed file parsing errors affecting DBF (FPT memos), PPT (headers and footers), and XLS ("Invalid packed Unicode sequence" error indexing an XLS file).

dtSearch 6.20 (Build 6320) Released May 6, 2003

Enhancements (dtSearch Desktop)

• Support for indexing Outlook 2003 messages. (Note: This is based on the Office 2003 Beta 2. It is possible that Microsoft may make additional changes to Office 2003 when it is released that will require more changes to the dtSearch Outlook indexer.)
• New search results list
  • The search results list can contain a brief synopsis for each item showing the first couple of hits and a few words of context around each hit. To enable this feature, click Options > Preferences > Search results, and check the box labelled "First hits in context".
  • Click the <-> mark in the upper left corner of search results to automatically size columns to fit their contents. Click it a second time to automatically size columns to fit in the search results window. (Click Options > Preferences > Search results to have search results automatically resized in either way.)
  • Right-click the <-> mark for quick access to search results format settings
  • Drag and drop column headers to change the order in which search results items appear
  • Items in search results can be dragged to Explorer (to move the files) or to email programs (to send the files as an attachment).
  • (To make dtSearch use the search results list from prior versions, run dtsearch.exe or dtsearchw.exe with the /lv command-line switch.)
• New text filtering option for indexing recovered forensic data. See the Options > Preferences > Filtering options dialog box and the "Filtering options" help topic in the dtSearch help file for more information
• New option in the Options > Preferences > File Types dialog box to require that a set of files be indexed as HTML or plain text, even if they appear to have a different format.
• Enhanced filename filters for use inside ZIP archives. See the "Filename Filters" topic in the dtSearch help file for more information.
• Press Ctrl+Shift+UP to enlarge the text font, or Ctrl+Shift+DOWN to reduce the text font.
• Edit > Copy File handles items in container files (such as ZIP archives or databases) better, copying only the items retrieved from the search rather than the entire container. For example, if a search retrieves sample.doc inside c:\archive.zip, then "Copy File" will extract and copy sample.doc rather than copying the whole archive.zip file.

Enhancements (dtSearch Web / dtSearch Publish)

• New ResultsTableItem macros: %%PhraseCount%% (number of hits in a document, counting each phrase as a single hit) and %%HitsByWord%% (list of words or phrases matched in a document, with the number of hits on each).
• New settings for dtsearch_options.html: HttpProxy and SERVER_NAME. See dtSearch_Web.chm for more information.

Enhancements (dtSearch Engine)

• Added ZIP file parser to Linux version
• Added Java JNI API to Linux version
• ExText text extracting algorithm, for indexing recovered forensic data, integrated into the dtSearch Engine as the "Filtered Binary" file parser.  See "Filtering options" in the dtSearch Desktop help file for information on how this filtering algorithm works.  A new value for the Options.binaryFiles flag, dtsoFilterBinaryUnicode (4), enables this parser.
• FileConverter and dtsFileConvertJob2 have new typeId property that can be used to specify the file parser to be used with the input.
• Serialized search results include search flags and fuzziness
• Added dtsoFfXmlHideFieldNames field flag to suppress indexing of field names in XML files.
• Data source API (C++): Added pFileInfo member to dtsDataSource to provide information on the last file indexed.  See "dtsDataSource" in dtengine.chm for more information.
• Data source API (C++): Added typeId to dtsInputStream to provide a way to specify the file parser that should be used for an input file.  See "dtsInputStream" in dtengine.chm for more information.
• Data source API (VB/ASP): Added DocId, DocWordCount, and DocTypeId properties of DataSourceToIndex to provide information the last file indexed.  See "Indexing ActiveX Data Sources" in dtengine.chm for more information.
• Data source API (Java): Added getDocId, getDocWordCount, and getDocTypeId methods in new DataSource2 class to provide information the last file indexed.  See the DataSource2 topic in the JavaDoc documentation for more information, and see the Java dsource sample application for sample code.
• Data source API (Java): Added getDocBytes() to provide a way to return documents in a memory buffer
• Java FileConverter and SearchReportJob objects: Added setDocBytes() for a calling program to provide an input document in a memory buffer
• Java API: Added SearchResults.getDocDetailItems(), returning the document properties as a java.util.Map
• Java API: Added SearchFilter object, SearchResults.serializeItemAsXml
• phraseCount, reporting the number of hits matched in a request with each phrase counted as a single hit, is computed if dtsSearchWantHitsByWord search flag is set. This value can be accessed in the C++ API as dtsSearchResultsItem.phraseCount, and in the COM and Java interfaces as DocDetailItem("_phraseCount").
• Added dtsSearchJob.maxFilesToRetrieve2 to provide a 32-bit version of this limit, and made the Java and COM maxFilesToRetrieve properties 32-bit.
• dtsSearchFilter (C++):  Added getIndexCount, getIndexPath, and getItems methods to extract information from a search filter.
• SearchFilter (Java):  Added getIndexCount, getIndexPath, and getItems methods to extract information from a search filter.

Fixes

• dten600.dll:  Improved PDF file parser handling of XObjects and corrupt but partially-readable data streams
• Spider did not cache HTTP sessions, causing problems with forms-based authentication
• dtSearch Web: "Next 25" paging did not work correctly when search results were sorted by a custom field
• Spider reported modified date for downloaded web pages based on the "Date" header rather than the "Last-Modified" header
• xfilter filename searches not including a \ could still match against the full path of a document
• dten600.dll: Check for "index full" condition uses improved method for estimating amount of space left to complete current update
• dten600.dll: A script bracked by <% and %> was incorrectly parsed if the script contained the <> comparison operator.
• Forix.exe: Forensic indexer did not apply exclude filters inside ZIP files, so the entire contents of a ZIP would be indexed without regard to any exclusions in the filters.
• dten600.dll: In an accent-insensitive index, combining diacriticals (U+0300-U+0362) were not removed
• dten600.dll: If a PDF file has a title attribute that is a path and filename, the path from the title attribute was reported in search results as the location

dtSearch 6.11 (Build 6276) Released December 24, 2002

Enhancements (dtSearch Desktop)

• Edit > Copy File can download web pages indexed by the spider and extract email messages, attachments, and other Outlook items, and copy them to the target folder
• Edit > Copy File provides detailed logging of file copy errors
• Added option to unconditionally suppress password prompts in the Spider
• New = wildcard character matches any single digit. Example: X=== would match X123 but not Xabc

Enhancements (dtSearch Web / dtSearch Publish)

• New %%Synopsis%% search results macro can be used to add a brief hits-in-context display to search results. See "The Options File" in dtSearch_Web.chm for details.
• Search results can be sorted by user-defined fields, and search forms can specify any combination of search flags to use when sorting.

Enhancements (dtSearch Engine)

• Added SearchReportJob.MaxContextBlocks, which provides a way to limit a search report to the first N blocks of context in a document
• Added SearchReportJob.MaxWordsToRead, which provides a way to limit a search report to the first N words in each document
provide two ways to limit a search report
• Added SearchFilter.GetItemArray and SearchFilter.GetItemArrayVBS, which provide a way to get an array containing doc ids selected in a SearchFilter
• Added SearchResults.AddDoc, which can add a specific document to a search results list by doc id
• Added C# sample web searching application
• Added dtsSortCleanText sort flags to remove leading punctuation, white space, "Re:", "Fwd:", and "Fw:"
• Added two new sample applications in C#: ADO.NET, demonstrating database indexing using ADO.NET, and asp_search, demonstrating a search function for a web site.

Fixes

• dten600.dll: External file parsers would not load from a dtSearch "Home" directory whose name contained accented letters
• mfcdemo sample application: two files were missing from the setup program
• dtindexer.exe: Spider did not identify user agent as "dtSearchSpider"
• dtindexer.exe: Intermittent crash indexing very large files stored in ZIP archives
• dtindexer.exe: Fixed several minor problems with RTF file parser (extra ; characters in text, incorrect handling of \emdash and \endash directives, \info properties)
• dten600.dll: MBOX file parser did not recognize Mozilla email archives
• dten600.dll: Fixed hit highlighting error in boolean field search with nested or/and condition (a or (b and c))
• dtindexer.exe: Fixed bug in index library manager that could cause indexer to crash at startup
• dten600.dll: Word wrapping in file conversion incorrectly handled words longer than 256 characters
• dten600.dll: Auto-detection of Unicode text files did not work with text files shorter than 20 characters
• dten600.dll: Items contained in ZIP archives indexed using the data source API did not inherit field attributes of the containing ZIP.

dtSearch 6.1 (Build 6260) Released November 3, 2002

Enhancements (All products)

• PDF indexing improvements: (1) Added support for indexing PDF files with security passwords (40-bit only); (2) better detection of word breaks; (3) indexing of annotations and form fields
• Improved formatting of Eudora messages
• Faster and more robust indexing of dBASE and FoxPro databases (ODBC is no longer used or needed for indexing these files)
• MBox message archive support. (Because MBox archives cannot be automatically detected, use the File Types dialog box to specify which files should be indexed as MBox archives.)

Enhancements (dtSearch Web)

• Added highlighting of hits on documents indexed using the spider (for example, third-party web sites)
• Added "Next 25" and "Prev 25" links to search results for paging through long search results lists
• Search form included in search results is initialized with previous search request

Enhancements (dtSearch Desktop)

• "All words" and "Any words" search options
• Option to retrieve most recent (vs. most relevant) documents
• Spider can index https (SSL) web sites.
• Spider can index password-protected web sites. Click Options > Preferences > Spider Options to set up passwords and security settings.
• Spider uses WinHttp 5.1 library under Windows XP and Windows 2000 SP3. This library provides improved handling of authentication and generally better performance (in our testing). WinHttp 5.1 is included with Windows XP and Windows 2000 SP 3 and is not available for other platforms.
• "Create Group Policy" dialog box added for automatic deployment of dtSearch and shared indexes across a network. See the "Automatic deployment of dtSearch on a network" topic in the dtSearch Desktop help file for more information.
• Outlook indexer can index appointments and journal items, in addition to messages, tasks, contacts, and notes
• Option to change the size of the dialog box font (Options > Preferences > Dialog box font size)
• Preferences dialog box reorganized
• Edit Noise Words dialog box

Enhancements (dtSearch Engine)

• Added dtsSortFloatNumeric sort flag
• New HttpSearch sample application demonstrates client-server searching with dtSearch Web on the server and the CHttpSearchJob C++ class used on the client to perform a search.
• SearchFilter object -- added SelectItemsBySearch method to allow a search filter to be set up as the results of one or more searches.
• pOnIndexWordFn added to the dtsIndexJob in the C++ API, to allow an indexing program to modify text as it is being indexed (to customize character handling or add alternative forms of a word to the index).  See the dtsOnIndexWordInfo topic in dtengine.chm for more information.

Fixes

• dten600.dll: Default name for alphabet file changed from "ENGLISH.ABC" to "DEFAULT.ABC", since it no longer has language-specific data
• dtindexer.exe: Merge indexes did not combine the list of folder selections for the merged indexes
• dtSearch.exe: Cursor keys sometimes did not scroll document in HTML viewer
• dten600.dll: Search report included an extra line of context before the hit
• dten600.dll: HTML hit highlighter highlighted an extra word if the search word ended in &nbsp;
• dten600.dll: PDF file parser incorrectly handled case where same font resource had different names in PDF file
• Setup program changes: (1) dtSearch Desktop and dtSearch Developer (Web/Engine/Publish) made into two separately installable products; (2) uses Windows Installer Service 2.0; (3) improved reboot handling under Windows 9x; (4) new "Repair" setup option to fix a damaged installation.
• dtSearch.exe: "fields" button in Search dialog box did not position the cursor correctly after fields were inserted into a search request
• dtindexer.exe: Spider reports when a web page cannot be accessed due to an authentication error or server error.
• dten600.dll: If an index path contains an accented letter, the noise word list was not used and all words were indexed.

dtSearch 6.07 (Build 6205) Released June 17, 2002

Enhancements (All products)

• Faster indexing for large indexes (typically about 20% faster)
• Added support for UCS-16 encoded HTML and XML (little-endian and big-endian)
• Blocks of HTML can be excluded from indexing using <!--BeginNoIndex--> and <!--EndNoIndex--> tags. (The comment tags must appear exactly they appear here, with no spaces or other variations.)
• Improved formatting in display of Excel spreadsheets

Enhancements (dtSearch Desktop)

• Option to automatically use "Report" view for very long text files
• Search reports can include documents indexed with the spider (http: references) and Outlook messages
• Edit|Copy file list to copy a list of filenames from search results to the clipboard
• Indexing Options: new option to "Index HTML scripts, styles, links, and comments" (causes these items to become visible and searchable in dtSearch)
• Improved status reporting in the Forensic Indexer (forix.exe).
• "View File" can be used to open a saved search results list

Enhancements (dtSearch Engine)

• dtsfclib.lib: updated to work with Visual Studio .NET. See "C++ Support Classes" in dtengine.chm
• C++ Support Classes can optionally be placed in a "dtSearch" namespace by declaring the USE_DTSEARCH_NAMESPACE macro
• SearchFilter object - added Read and Write functions to save to/read from disk files, ReadMultiple to read and combine multiple filters, and AND and OR operations to logically combine two filters
• Added percentFull property in dtsIndexInfo
• An alternative search syntax, the "All Words"/"Any Words" syntax, can be used in searches using the dtsSearchTypeAllWords and dtsSearchTypeAnyWords search flags. This new search syntax supports use of quotation marks to indicate phrases and + and - to indicate required and excluded words. See "dtsSearchTypeAllWords and dtsSearchTypeAnyWords" topic in dtengine.chm.
• andany search connector added to allow optional words to be added to a search request. (see "Search Requests" help topic in dtengine.chm)
• IndexJob object -- added GetIndexInfo method to access to index properties in the COM interface
• IndexJob object -- MaxMemToUseMB property provides a way to limit the amount of memory used during indexing
• For forensic applications, new FieldFlags options to allow indexing of hidden HTML content: dtsoFfHtmlShowLinks, dtsoFfHtmlShowImgSrc, dtsoFfHtmlShowComments, dtsoFfHtmlShowScripts, dtsoFfHtmlShowStylesheets, dtsoFfHtmlShowMetatags.

Enhancements (dtSearch Web)

• MaxWordsToRetrieve setting added to dtsearch_options.html, providing a way to limit server resources consumed by general searches like "a* or b* c*"
• BooleanConnectors setting added to dtsearch_options.html, so that the boolean connectors can be customized on web forms
• FieldFlags setting added to dtsearch_options.html
• OriginalSearchForm CGI variable added to search forms as backup to prevent broken link to options file
• dtSearch Web Setup can add a "Search Type" option to search forms, allowing the user to select between "All words", "Any words", "Exact phrase", and "Boolean" search types

Enhancements (dtSearch Publish)

• CD Wizard can automatically generate a CD using either Apache or Microweb

Enhancements (dtSearch Engine for Linux)

• dtSearch Engine for Linux (libdtsearch.so.6.0) updated to dtSearch 6.07 code base (build 6200) and built with GCC 3.1.

Fixes

• dten600.dll: VB data source API made the creation date the same as the modified date
• dten600.dll: Memory leak in DSearchReportJob class (affected COM and Java APIs)
• dtSearch.exe: if document opened in Quick View Plus embedded in browser, could not then open another document as HTML
• dtSearch Web: Top and First Hit buttons did not work with Netscape 6.2
• dten600.dll: Search reports output as Ansi text could contain character 0x1e, causing truncation of reports in applications that treat this character as EOF
• dtSearch Publish: Fixed errors displaying non-English characters in startup CD dialog box
• dten600.dll: Fixed error parsing fields identified by HTML comments
• dten600.dll: Fixed error in IndexGetDocInfo C++ API function that caused it to fail to recognize a valid open index
• dten600.dll: xfilter Filename searches for names containing the stemming char (~) did not work
• dten600.dll: Improved logic for handling noise words in phrase searches, when the noise word in the search request matches both noise words and words that are not noise words
• dten600.dll: Fixed bug in ActiveX Data Source API in build 6204

dtSearch 6.06 (Build 6173) Released March 15, 2002

File Format Enhancements (all products)

• Automatic detection of encoding type for plain text and HTML files without Content-Type META tags, based on language of contents. Supported languages/encodings: Arabic, French, German, Greek, Hebrew, Italian, Russian (KOI_8R and CP1251), Spanish, Central European (CP1250), and UTF-8.
• .CSV support enhancements: Automatic detection of delimiter (tab, semicolon, or comma) and encoding (Unicode, UTF-8, or Ansi)
• Filename-only indexing of non-text formats: asf, ani, avi, mpg, mov, jpg, tif, gif, png, cab, ix, chm, ttf, pst, wav, mp3, bmp, wmf, emf. (These formats are detected by checking the file contents, not just the extension, so a Word document named "something.jpg" will still be indexed as a Word document.)
• Added support for: .MHT archives (single-file web pages saved by Internet Explorer)
• Added support for: .EML files (emails saved by Outlook Express)

Enhancements (dtSearch Desktop)

• Default character encoding option setting in the Options > Preferences File Types dialog box, provides a way to override automatic encoding detection for file types that do not contain encoding information
• extext.exe text extractor utility for extracting text from large binary files (click Start|Programs|dtSearch|Tools|Extext)
• forix.exe forensic indexer utility, for creating indexes from very large volumes of data (click Start|Programs|dtSearch|Tools|Forix)
• Optional limit on the maximum amount of text to display from a file (in Options, Preferences, Other Options)

Enhancements (dtSearch Engine)

• WordsOfContextExact search report property (dtsReportByWordExact flag in C++ API) allows search report to contain exactly the requested number of lines of context around each hit
• Fields returned through the data source API can be indexed without the field name being searchable. To suppress indexing the field name, add * in front of the name
• C++ Support Classes are included in static libraries (in addition to the source code previously included) for easier use in Visual C++ projects
• C++ data source API supports error reporting by the caller's data source
• SetSortKey() method allows sorting search results on custom values
• dtsSearchAutoTermWeight search flag allows automatic term weighting (as in natural language searches) for boolean query
• Search filters provide a way to pre-select the documents that are eligible to be returned in a search. This feature can be useful when a database search and a dtSearch text search must be combined. See the SearchFilter (VB) or dtsSearchFilter (C++) topic in the dtSearch Engine help file for details.
• dtSearch Web (dtisapi6.dll) logging search requests and document access. For more information, see the "Generated Files" topic in dtSearch_Web.chm.

Enhancements (dtSearch Engine for Linux)

• dtSearch Engine for Linux (libdtsearch.so.6.0) updated to dtSearch 6.05 code base, and PDF indexing support added.

Fixes

• dten600.dll: "Stored" fields limited to 512 characters
• dtindexer.exe: Failed to convert Outlook message dates to UTC when indexing, so message time shown in search results was off by a few hours
• dtsjava.dll: Java JNI wrapper reported file dates through search results using 1-12 for the month, instead of 0-11, so the reported month was off by one.
• dtimage.exe: View as Image did not check for external viewer settings
• dtimage.exe: Draw errors updating image when page changes with multiple .TIF files
• dtindexer.exe: Failed to initialize Outlook indexing if user has more than one profile, and Outlook is set to prompt for the profile to use at startup
• dtindexer.exe: Incorrectly handled indexing Outlook message folders that contain / or \ in the folder name
• dtsearch.exe: Displayed internal Outlook name (a long number) for some messages, rather than the subject.
• dten600.dll: "<![endif]-->" displayed incorrectly in some MS Word documents saved as HTML
• dten600.dll: In plain-text documents that contain exactly one line of text, the title attribute was doubled

dtSearch 6.05 (Build 6146) Released November 29, 2001

Enhancements (dtSearch Desktop)

• Option setting, in Options, Preferences, Other Options (formerly "Index Defaults"), to change the fonts in the Search dialog box
• Improved and simpler Index Library Manager. See the "Index Library Manager" topic in the dtSearch help file for more information.
• More responsive switching between files when a long file is being opened
• Outlook Express 5/6 (*.dbx) message store indexing (including attachments)
• Unindexed Search for List of Words

Enhancements (dtSearch Engine)

• Outlook Express 5/6 (*.dbx) message store indexing (including attachments)
• dtsSearchWantHitsArray flag, with dtsSearchWantHitsByWord, provides word offsets of phrases

Enhancements (dtSearch Publish)

• cdrun.exe can check for required software and install dependencies as needed or display an error page. See the "Software Dependencies" topic in dtSearch_Web.chm for details.

Fixes

• dten600.dll: dtsSearchWantHitDetails report field names incorrect in multi-index search
• dtimage.exe: Print problems (incorrect scaling to fit page, only printed one page at a time) when printing image files
• dtSearch.exe: Search for List of Words, "Boolean" option changed to "One boolean expression per line"; improved error reporting

dtSearch 6.04 (Build 6126) Released October 15, 2001

Enhancements (dtSearch Desktop)

• New "Whole file" search report option
• MaxWordsToRetrieve default value increased to 64k, and registry entry added to allow for customization
• Added Index Library Manager option to suppress check for index libraries in UserData and dtSearch program folders
• Optional checkboxes in search results list (use Options|Preferences|Search Results Format to enable them) for easier selection of multiple items
• Edit|Select all and Edit|Clear selections commands for easier selection of items in search results
• "Search for List of Words" search type in the Search menu, for searching using the contents of a text file as the search request. Results of the search can be displayed in dtSearch Desktop or can be written to a text file.
• Help|About dialog box shows configuration details
• Outlook indexing problems created by the Microsoft Outlook Security Update resolved.
• Outlook contact details (phone, fax, company, title, etc.) indexed
• Outlook attachments indexed
• Select Outlook Folder dialog box allows for automatic selection of subfolders
• File Segmentation Rules enhanced to support headers on segments (so HTML and XML files can be segmented)
• Scheduled Index Updates can update more than one index in a single task
• Hits are centered in the dtSearch window instead of appearing at the end

Enhancements (dtSearch Publish)

• dtSearch Publish, included in the dtSearch Engine download, provides tools and a setup wizard for running dtSearch Web from a CD.

Enhancements (dtSearch Engine)

• dtsearch_paging.asp sample added to demonstrate display of search results in pages
• dtsearch.asp sample uses improved JavaScript for hit navigation
• New properties in Options object in the COM and Java JNI interface: macroChar, stemmingChar, fuzzyChar, synonymChar, phonicChar, weightChar
• New properties in FileConverter object in the Java JNI interface: baseHref, alphabetLocation

Enhancements (dtSearch Web)

• Added AutoStopLimit setting in Form Builder
• Added MaxUrlSize setting in _options.html file to limit the maximum size of a URL created by dtSearch Web in search results.
• Relevance score automatically reported in dtSearch Web searches (this change only affects search forms created with the new dtSearch Web Setup)

Fixes

• dtSearch.exe: Error message "Failed to update the system registry. Please try using REGEDIT." when dtSearch is launched, due to MFC bug described in Q254957
• dtSearch Web: Incorrect hit highlighting with hyphens set to "searchable"
• dtsearch.asp: Fixed encoding problems in sample code for display of Unicode text
• setup: installer modified to work around bug in Windows XP's implementation of msiexec.exe
• dten600.dll: Character encoding error in MS Word 95 document
• dtv_odbc.dll: Extra blank lines displaying database records
• dtsviewr.h: dtsInputSource and dtsInputSourceReader renamed to dtsInputStream and dtsInputStreamReader (with a #define to preserve backward-compatability)
• dten600.dll: PDF file parser error reading xref table from PDF 1.0 document
• dten600.dll: Unindexed search did not delete file list temp file if searching data source or nonexistent files
• dtSearch.exe, dtSearch Web: Sorting search results by filename did not check, for PDF/HTML files, whether the filename or the Title was currently displayed as the filename. As a result, the list was always sorted by the title for these file types.
• dten600.dll: On a sort by Location, items within a particular location were sorted in ascending order by score, rather than descending order
• dten600.dll: Fixed error handling <style> tags in HTML
• dten600.dll: Fixed error handling <src> tags with blank quoted ALT attribute in HTML
• dtsearch.exe: Index Create dialog box did not browse for UNC folders correctly
• dten600.dll: Error searching xfilter expression for string containing ":"
• dten600.dll: Incorrect handling of &quot; expression in HTML and XML tag attributes
• dten600.dll: IndexJob object in the COM interface -- StatusBytesToIndexKB value was not set
• dtWebSetup.exe: Recovers partial virtual path information when metabase access functions throw an exception
• dtWebSetup.exe: Wrote closing title tag of search form incorrectly
• dtSearch.exe: Better trapping of JavaScript errors when navigating hits
• dtSearch.exe: Fixed problems with space, backspace, enter, and tab hotkeys when reviewing search results
• dten600.dll: Fixed error in WordPerfect file parser character tables for processing some Arabic and Greek characters
• dtSearch Web: JavaScript for "Next Hit" button did not find last hit in a document
• dten600.dll: Report generated by dtsSearchWantHitDetails flag had hits in descending instead of ascending order
 

dtSearch 6.03 (Build 6079) Released June 1, 2001

Enhancements (dtSearch Desktop)

• dtSearch.exe can run from a shared network folder (see the help file for details)
• Much faster loading of long documents
• Loading of long documents can be cancelled
• Option setting not to automatically open the first document
• Hit highlighting shows which hit is currently selected
• Edit|Copy File added
• New UTF-8 file type in File Types dialog box
• User-defined fields can be included in search reports and search listings
• New dtsrun.exe launcher to select dtsearch.exe or dtsearchw.exe automatically

Enhancements (dtSearch Engine)

• New CD Wizard for setting up dtSearch Web to run from a CD. [This new option for CD publishing will not be released with version 6.03 due to the need for more beta testing. It is included in 
  pre-release form for testing and evaluation purposes only.]
• Added outputStringMaxLen value and dtsListTabDelimit flag for dtsListIndexJob
• Added to Java API: urlEncodeItem, urlDecodeItem, serializeToXml, serializeFromXml, getHitByteOffsets, getHitBytePageParaOffsets
• Added dtsUSearchAccentSensitive flag to make unindexed searches accent sensitive (default changed to accent insensitive)

Enhancements (dtSearch Web)

• Improved hit navigation JavaScript
• Next/Prev Hit buttons provide better information messages when PDF files or files without hits marked are displayed

Fixes

• dtwebsetup.exe: Did not display default search feature selections correctly in Search Form Builder
• scriptrun.exe: Did not check for dtSearch Desktop registry settings
• dtsearch.exe: pressing Ctrl-Y with no search results open would cause a crash
• dten600.dll: WordPerfect character tables incorrectly encoded Unicode values for some Greek letters
• dten600.dll: Bug in dtssMapHitsInFile caused word counting to be off due to error reading alphabet
• dtindexer.exe: Clicking "Delete Task" in Schedule Updates deletes the task, even if you answer No to the question
• dtsearch.exe: Hit navigation did not work with the Internet Explorer 6 beta
• dtsearch.exe: Detect and handle Adobe Reader 5's "Certified Plug-ins Only" option setting
• dten600.dll: More efficient memory use when indexing very long files
• dten600.dll: When truncating words longer than the maximum word length, count Unicode characters rather than UTF-8 characters
• dten600.dll: Some Hebrew vowel marks not classified as letters in Unicode tables
• dtisapi6.dll: Error handling blank fields on search form
• dten600.dll: Crash indexing damaged Excel spreadsheet embedded in Word document
• dten600.dll: Crash compressing an index
• dtwebsetup.exe: JavaScript called by NextDoc crashed Adobe Reader
• dten600.dll: Null character in damaged PDF file's xref table crashed PDF file
• dten600.dll: Excel file parser incorrectly handled "extended strings" stored by Far East version of Microsoft Word

dtSearch 6.02b (Build 6055) Released April 9, 2001

• dtsetup.exe: fixed "Wrong OS Version" message 

dtSearch 6.02 (Build 6055) Released April 2, 2001

• dtsearch.exe: Unindexed search crashes attempting to display the "Index" column in search results. 
• dtsearch.exe: SPACE and BACKSPACE do not work in HtmlHelp when launched from dtSearch. The reason for this is that dtSearch hotkeys are still active when help is launched, and in dtSearch SPACE = Next Hit and BACKSPACE = prev hit. 
• dtsearch.exe: Search dialog box, under Windows 98, does not move the cursor to the end of the search request when the user clicks AND/OR/etc. buttons
• dtsearch.exe: Sort by file type does not do anything
• dtsetup.exe: Installing under Whistler (Windows XP) beta causes an MSI error.
• dtsearch.exe: After launching a file to view it in search results, the dtSearch window moves in front of the launched program
• dtsearch.exe: Pressing Ctrl-M ("View as Image") with no search results displayed crashes dtSearch
• dtsearch.exe: occasionally displayed plain text documents right-justified
• dtsearch.exe: intermittent crash switching back to dtSearch window if Search dialog box not initialized yet
• dtsearch.exe: very intermittent right-justification of plain-text documents in viewer window
• dtsearch.exe: Select All button in Search dialog box did not work 
• dtsearch.exe: Problems displaying field names in Browse Words dialog box
• dtsearch.exe: "Include word counts" checkbox in List Index dialog box did not work
• dtsearch.exe: search dialog box warns when unindexed searching is left on, and when file filters are left over from a previous search under More Search Options. The warnings can be suppressed with a checkbox, and there is a new Options|Enable warnings... command to enable all optional warnings.
• dtsearch.exe: HTML error caused extra underlining in some documents when highlighting hits
• dtsearch.exe: Change to booleanConnectors setting did not work if boolean connectors string was too long
• dtsearch.exe: "Search in a new window" did not do anything under Windows 9x
• dtindexer.exe: warns when the "Clear index before adding documents" box is checked and the index already has data (like 5.25 did -- people have been asking why dtSearch fully rebuilds their indexes each time)
• dten600.dll: Word documents created by a Japanese version of Word 95 were not recognized
• dten600.dll: HtmlTitle field sometimes not generated for contents of the HTML <TITLE>
• dten600.dll: crash indexing a corrupt WordPerfect document
• dten600.dll: SearchReportJob sometimes put a line break between the beforeHit mark and the first letter of the hit word
• dten600.dll: ComFileConverter failed if InputFile left blank.

dtSearch 6.01 (Build 6048) Released March 14, 2001