In Warehouse

OmniAI's In Warehouse offering allows for data transformations directly within your existing data warehouse. This method is recommended for organizations seeking a streamlined architecture, enhanced access control, and superior query performance.

It supports transformations within Snowflake, PostgreSQL, MySQL, and MongoDB, enabling the writing of transformed values back to new tables in the same source.

You will need to create an OmniAI database user following these steps.

How It Works:

1. Data Reading and Writing: OmniAI reads values from your specified data source and writes the transformed values back into a new table within that same data source.

2. Simplified Architecture: This approach minimizes the complexity of your data architecture by keeping all operations within the same warehouse, avoiding the need for external data movement.

3. Access Control and Security: Operating within your warehouse allows for better utilization of existing access control mechanisms, ensuring that data remains secure and governance policies are upheld.

4. Improved Performance: By keeping data transformations within the warehouse, OmniAI leverages the native performance optimizations of your data storage system, ensuring faster query times and more efficient data processing.

Example Use Case:

• A user selects the customer_reviews table and activates a sentiment analysis transformation. OmniAI then creates a new table, omni_customer_reviews, within the user's database. This table includes:

  • customer_reviews_id: A foreign key reference to the original customer_reviews table.

  • sentiment: The result of the sentiment analysis transformation.

Configuration Options:

• Source Table: The table from which data is read for transformation.

• Transformation Type: The specific transformation to apply (e.g., sentiment analysis).

• Destination Table: The new table where transformed values will be stored. This table is automatically named but can be configured in certain scenarios.

• Column Mapping: Define how source columns map to destination columns, including any renaming or exclusion of columns not needed for transformation.

• Primary Key Handling: Ensures that the primary key from the source table is preserved to maintain data integrity and relational context.

Available For:

• Snowflake

• PostgreSQL

• MySQL

• MongoDB

Important Note: The In Warehouse option is not available for HTTP Request and CSV Upload data sources. These types of data can only be processed using the Hosted DB option provided by OmniAI.