Using the dtSearch Engine on Azure and AWS

General information on using the dtSearch Engine with Azure and AWS, including platform requirements, index storage and and data storage.

CodeProject article:  "intelligent search" with dtSearch Engine and Azure Storage

Article Excerpt:

   
CodeProject article:  "intelligent search" with dtSearch Engine and Azure Storage

Azure functions are an extremely versatile platform to operate on. The dtSearch tools allow us to build and manage a search index anywhere that we need intelligent search capabilities ...

What if there was an easier way to store my documents in the cloud for backup purposes and get the same great searching capabilities as when I have those documents stored locally? That’s where dtSearch and Azure functions can together provide a neat experience for managing your documents and their contents in the cloud ...

Azure Storage is a great facility that keeps multiple backups of your documents and data across regions ... Azure Functions allow us to write a little bit of code that can be managed and run when appropriate to interact with those storage locations and perform services for us ...

In the case of dtSearch and our data, this is a perfect match. We can structure our storage location so that it properly houses the archive of documents we want to process with our search engine, and we can also trigger the regeneration of that search index appropriately when a new document is added to the collection. The Azure Function architecture also allows us to make querying that managed search index a trivial operation with an HTTP endpoint, accessible from anywhere in the world.

Full Article


CodeProject article:  "The article brings together the Microsoft Azure Cloud with dtSearch to enable secure hosting of terabytes of a broad spectrum of data types combined with instant dtSearch search access from nearly any computer or device"

In this .NET solution, all dtSearch components (including indexes) are cloud-based.

Article Excerpt:

   
CodeProject article:  "The article brings together the Microsoft Azure Cloud with dtSearch to enable secure hosting of terabytes of a broad spectrum of data types combined with instant dtSearch search access from nearly any computer or device"
The Microsoft Azure Cloud makes available more data centers than Amazon and Google combined, along with 3X data replication and the most advanced security practices in the industry ...

dtSearch uses its own built-in “document filters” to instantly search terabytes of a broad range of data types: “Office” documents, PDFs, emails with multilevel attachments, web data, and other databases. dtSearch offers over 25 different search options, including faceted searching and multiple advanced data classification options, as well as search results display with highlighted hits ...

Azure Files stores the dtSearch indexes. Azure Files lets you share data across any number of applications through the SMB (Server Message Block) protocol.

Full Article


CodeProject article: “Blazing Fast Source Code Search in the Cloud” using the dtSearch Engine under Microsoft Azure

Article Excerpt:

   
CodeProject article: “Blazing Fast Source Code Search in the Cloud” using the dtSearch Engine under Microsoft Azure
Using dtSearch and the techniques in this article will make your data searches lightning fast, making it possible to search terabytes of data with sub-second response time.

But first, two preliminary notes about this blog post. (1) The blog post describes source code data, but the same approach would apply to other data stored in the Microsoft Azure cloud: HTML, XML, MS Office documents -- even email data. (2) While the data in this blog post resides in the Microsoft Azure cloud, the indexes are on a local PC. A subsequent article will address data and indexes in the cloud.

Here is a workplan of our overall project:
Project Workplan Part 1: Portal Work and Part 2: Visual Studio WorkProject Workplan Part 3: Visual Studio Work and Part 4: Test and Release

Full Article


CodeProject article: “Windows Azure SQL Database Development with the dtSearch Engine”

Article Excerpt:

    CodeProject article: “Windows Azure SQL Database Development with the dtSearch Engine” Windows Azure SQL Database does not support the CONTAINS keyword. CONTAINS is important because it supports searches for precise or fuzzy (less precise) matches to single words and phrases, words within a certain distance of one another, or weighted matches. CONTAINS is a predicate used in the WHERE clause of a Transact-SQL SELECT statement.

But there are ways to get this functionality with the help of third party software. I recently downloaded all the movie information for films that took place in San Francisco. The San Francisco government website provided this interesting information that I wanted to search through very quickly. The data wasn't very normalized so there are many columns that are difficult to search. I downloaded this information and imported it into a database. Specifically I used SQL database hosted in the Windows Azure platform.

My goal was to be able to find records in the database table, searching in all columns and in all rows quickly and efficiently. Essentially, I wanted to build one master index that would allow me to perform the search locally against the generated index and then be able to pull up the specific record in the database table once I found what I was looking for.

dtSearch is a product that allows you to accomplish this goal. I downloaded it and installed it. Of course, I use it for many of my other searching needs as well. dtSearch has a variety of products. This post will focus on how you can index a database table so that you can perform lightning quick full-text searches.

Full Article

dtSearch Engine: key features