Skip to main content

Configuration

When using the VulcanSQL and creating the project through vulcan init, you will see a file named vulcan.yaml under the project root.

The vulcan.yaml is the VulcanSQL project config, a lot of features (e.g: Data Source, Authentication, API Document ..), and the core feature rely on the file, we will introduce it in this chapter.

Filename of Project Config

Although the vulcan.yaml is the VulcanSQL project config name, but it is not mandatory actually. You could rename the project config file, but VulcanSQL will default use it when you play our command e.g: vulcan build, vulcan serve, or vulcan start.

So for example, if you rename the project file name to hello-world.yaml, then when you run the above command, you should add the optional options -c or --config :

$ mv vulcan.yaml hello-world.yaml
$ vulcan start -c ./hello-world.yaml

Structure of Project Config

It's a default project config when you run vulcan init , you will see some config options you have defined, and we will let you know our config structure follow by the below default sample:

# The project name, description version
name: my-first-vulcan-project
description: A starter Vulcan project
version: 0.2.0

# The config options of core feature
template:
  provider: LocalFile
  folderPath: sqls
  codeLoader: InMemory
artifact:
  provider: LocalFile
  serializer: JSON
  filePath: result.json
schema-parser:
  reader: LocalFile
  folderPath: sqls
document-generator:
  specs:
    - oas3
types:
  - RESTFUL
# The external extension modules you wouldl like to use in your Vulcan project
extensions:
  duckdb: '@vulcan-sql/extension-driver-duckdb'

# The config options of built-in extensions
profiles:
  - profile.yaml
rate-limit:
  options:
    interval:
      min: 1
    max: 10000
enforce-https:
  enabled: false
auth:
  enabled: false
response-format:
  enabled: true
  options:
    default: json
    formats:
      - json
      - csv

You will see a split of 4 sections by the above comment, and the 4 sections combine our project information, core feature config options, built-in extension, and external extension config options.

Project information​

In the VulcanSQL project config, we will provide a section for you to define the project name and give the description for recording it so you or your partner could know the project's purpose. The version will record what VulcanSQL version you used, basically your installed version.

name: my-first-vulcan-project
description: A starter Vulcan project
version: 0.2.0

Config of Core feature​

VulcanSQL has some config options related to core features, below are the core feature options:

template:
  provider: LocalFile
  folderPath: sqls
  codeLoader: InMemory
artifact:
  provider: LocalFile
  serializer: JSON
  filePath: result.json
schema-parser:
  reader: LocalFile
  folderPath: sqls
document-generator:
  specs:
    - oas3
  folderPath: .
types:
  - RESTFUL
profiles:
  - profile.yaml

Actually, the template, artifact, and document-generator options are related to the vulcan build and vulcan serve command and we called build time and serving time, in VulcanSQL Team. For the schema-parser, types and profiles related to serving time,

Build time and Serve time​

In the build time, VulcanSQL will use the template options to find where is the user-written SQL files and compile the SQL files to the artifact compiled files to the artifact options defined places.

In the serving time, VulcanSQL will use the schema-parser options to find where is the user-written API schema files, then VulcanSQL will check the API protocol user would like to create the API endpoints according to the types options.

When API is created and users send requests to the API endpoints, VulcanSQL will use template and artifact options again for running the SQL files and analyzing them to send data sources, for the data sources connection data, VulcanSQL uses the profiles options to find each data sources settings.

template options​

  • provider - The provider represents what the provider used to read the SQL files. VulcanSQL uses the LocalFile type as a default value, which means the SQL files are put in local places.
  • codeLoader - The codeLoader option tells the compiler what code loader type we keep the data in, and VulcanSQL default use InMemory .
  • folderPath - When the provider is LocalFile, we need to set the folderPath option. The folderPath indicates a folder location which used to put your SQL files.
template:
  provider: LocalFile
  folderPath: path/to/folder
  codeLoader: InMemory

If you put to provide the folder name directly, we will find the folder in the current project.

folderPath: sqls # path to ./sqls folder

artifact options​

  • provider - Similar to template options' provider, but it used to save and read the artifact file. The default value is also for LocalFile
  • serializer - Indicate the format which the compiled artifact file serializes it.
  • filePath - The compiled artifact file kept location.
artifact:
  provider: LocalFile
  serializer: JSON
  filePath: path/to/file.json

schema-parser options​

  • reader - Similar to template options' provider The default value is LocalFile.
  • folderPath - When the provider is LocalFile, we need to set the folderPath option. The folderPath indicates a folder location which used to put your API schema files.
schema-parser:
  reader: LocalFile
  folderPath: sqls # Path to ./sqls folder

We suggest you put each API schema file in the same layer as the SQL file and name it as the same SQL file to recognize it easily, otherwise if your API schema does not define some field, VulcanSQL will use the API Schema filename to match the same name SQL file for querying data result:

/sqls
  # Query order SQL file and its API schema file
  - orders.sql
  - orders.yaml
  # Query users SQL file and its data API schema file
  - ursers.sql
  - users.yaml

For the API schema file Configuration, we will talk more in Data API Schema.

document-generator options​

  • specs - The API document used specifications, The document generator will generate the specifications and make our document server run it. The default is Open API 3.0 specification.
document-generator:
  specs:
    - oas3

Config of Built-in Extension​

VulcanSQL provides the extension components to let users could extend it for making the VulcanSQL powerful, and VulcanSQL also uses the extension components to create built-in extensions which enable it to run the feature.

Like the below configs, The profiles, rate-limit, enforce-https, response-format, and auth are our built-in extensions. These extensions work in serve time to make the API request have more restrictions when sending requests.

profiles:
  - profile.yaml
rate-limit:
  options:
    interval:
      min: 1
    max: 10000
enforce-https:
  enabled: false
auth:
  enabled: false
response-format:
  enabled: true
  options:
    default: json
    formats:
      - json
      - csv

Although the above has shown 3 default configs, actually not only these built-in extension configs, you do not see here because VulcanSQL uses the default value of other extensions which not appeared it.

Otherwise, each extension has different config options, so we will talk about each extension in detail and in each extension chapter.

For the above config, if you are interested, you could see Data Source Profile, Rate Limit (TBD, link), Enforce HTTPS (TBD, link), Response Format(TBD, link), and Authentication (TBD, link) first.

Config of External extension​

Besides built-in extensions, users could extend our extension components to create extensions, and these extended extensions are called external extensions.

Otherwise, VulcanSQL also has some external extensions which we have prepared, so if you would like to use external extensions, you could follow the below structure to define:

# Declare you would like to used-external module and give a module name
extensions:
  ext-module-name1: @vulcan-sql/xxx-module # installed Vulcan module
  ext-module-name2: path/to/folder/of/package # path

ext-module-name1:
  ... # The config options of the ext-module-name1

ext-module-name2:
  ... # The config options of ext-module-name2

# The config options of core feature and built-in extension
...

The extensions config could let user could declare what external extensions they would like to use in the VulcanSQL project.

The user should declare a variable called module name and specify it external extension location. The location has two ways to specify:

  • module name
  • local module file path.

Specify the module name​

you could use npm or yarn to install a 3rd-party source or VulcanSQL-provided external extension package ( @vulcan-sql/xxx-ext-module format ) and specify the package module name, e.g: ext-module-name-1 :

extensions:
  ext-module-name1: @vulcan-sql/xxx-module # installed Vulcan module

For example, if you would like to use @vulcan-sql/dbt it looks like the below:

extensions:
  dbt: '@vulcan-sql/extension-dbt'

dbt:
  modelFiles: path/to/folder

Specify the local module file path​

If you would like to use your local developed external extensions, you could specify the module path e.g: ext-module-name-2 :

extensions:
  ext-module-name2: path/to/folder/of/package # path

# Set the config options of the ext-module-name2 if ext-module-name2 provide config to let user could set.
ext-module-name2: ...

Not all external extensions have config options​

Not all external extensions have config options, some options maybe rely on internal or built-extensions config options and set options under internal or built-extensions config options. We should check each document of the external extensions module.

For example, if you would like to use @vulcan-sql/duckdb it looks like the below:

extensions:
  duckdb: '@vulcan-sql/extension-driver-duckdb'

But the duckdb is an data source driver, so if you would like to make your Data API /orders could query orders from duckdb driver, you should define the duckdb data source driver in data source profile like below:

# duckdb-profile.yaml
name: duck
type: duckdb
connection:
  persistent-path: ./test-data/moma.db
  log-queries: true
  log-parameters: true
allow: '*'

Or define it directly under the profiles config options of built-in extensions with extensions options in the project config:

# vulcan.yaml

# other config options core feature
---
extensions:
  duckdb: '@vulcan-sql/extension-driver-duckdb'

profiles:
  - name: duck
    type: duckdb
    connection:
      persistent-path: ./test-data/moma.db
      log-queries: true
      log-parameters: true
    allow: '*'

---
# other config options built-in extensions