SQL Plugins
DataJunction has a flexible SQL transpilation plugin system that enables you to add support for transpiling SQL queries across any SQL dialects. By default, support is enabled for Spark SQL, Trino, and Druid. You can also add custom transpilation plugins by following the instructions below.
1. Create the Plugin Class
Subclass SQLTranspilationPlugin
, and register it with the dialect(s) it supports using the @dialect_plugin decorator:
@dialect_plugin("customdb")
class CustomDBTranspilationPlugin(SQLTranspilationPlugin):
package_name: str = "customdb-sql"
def transpile_sql(
self,
query: str,
*,
input_dialect: Optional[Dialect] = None,
output_dialect: Optional[Dialect] = None,
) -> str:
import customdb_sql # Ensure the library is installed
# Ex: use the custom library’s transpilation interface
return customdb_sql.transpile(query, source=input_dialect.name, target=output_dialect.name)
2. Update Configuration
Modify your settings to activate the new plugin:
sql_transpilation_plugins = ["default", "sqlglot", "customdb-sql"]
3. Include Any Required Dependencies
Make sure the transpilation library for your dialect (e.g., customdb-sql) is listed in your requirements file and installed in the deployment setup.
4. Setup Engine + Catalogs
Once the plugin for your customdb
dialect is registered, you can create an engine with the dialect customdb
:
curl -X POST http://localhost:8000/engines -d '{"name": "customdb", "version": "", "uri": "", "dialect": "customdb"}'
Add this new engine to your existing catalog(s):
curl -X POST http://localhost:8000/catalogs/default/engines -d '{"name": "customdb", "version": "", "uri": "", "dialect": "customdb"}'
5. Usage
Now it is ready to use – you can request SQL in your new customdb
dialect!