Treasure Data's primary idea portal. 

Submit your ideas & feature requests directly to our product requirements team! We look forward to hearing from you.

Support parameters in cli queries


When we run a td query from a file using the "-q" option it would be great to pass in parameters rather than having to code a variation of the file for every query.

As a simple example:

If my query file looked like this:


  my table
  time between {start} and {stop}


Then we might run:

td query -w -c -T presto -d mydatabase -q myquery.tdql -f csv -o myqueryoutput.csv --start 123456 --stop 123789


td query -w -c -T presto -d mydatabase -q myquery.tdql -f csv -o myqueryoutput.csv --parameters start=123456,stop=123789


There are many use cases where we run the same query from the command line with different parameters.  (particularly when we are building data pipelines)  At this time we have to maintain a separate "tdql" file for every parameter we might run, and then have to identify the correct file at run time.   With that architecture, if we make a simple change to the query, we have to remember to update every file for each of the possible parameter values.

We could possibly wrap the td query statement with a bash script that does the parameter substitution and writes a temporary file with the actual query in it, and then execute the temporary file, but that can have its own risks and issues.  It would be much better if the td query function simply supported parameters.

[ Even better would there to be a way to store these queries at Treasure Data so the results could be pre-fetched or indexed or in some way accelerated to be more at the ready when we need to run them. ]





  • Rick Otten
  • Aug 9 2016
  • Under Review
  • Rodrigo Assis commented
    February 13, 2017 18:51

    It would also be good that this feature is done not only through CLI but also in the UI. Just like it currently prompts for TD_SCHEDULED_TIME.