Ideas

Treasure Data's primary idea portal. 

Submit your ideas & feature requests directly to our product requirements team! We look forward to hearing from you.

[DataConnector] Override yml file parameter by specifying option when run command issued.

Override yml file parameter by specifying option when run command issued.
Command option flexibility helps integration with current system.

  • Ryutaro Yada
  • Jun 10 2016
  • Already exists
  • Ryutaro Yada commented
    June 10, 2016 23:34

    Instead of implementing each command option for dataconnector, I think there is another approach to use benefit of liquid template engine feature which is supported from embulk 0.7.0(http://www.embulk.org/docs/release/release-0.7.0.html).

    For example, if we can use liquid file (td-bulkload.yml.liquid) which include {{ env.LOADFILENAME }}, we can specify each file at runtime as following.

    $ LOADFILENAME=xxxxxx td connector:issue td-bulkload.yml.liquid

    That way give us strong flexibility and make it easy to manage bulkload in batch program.

  • Mengke Li commented
    September 14, 2016 23:30

    Will be supported in conjunction with Treasure Workflow