Ideas

Treasure Data's primary idea portal. 

Submit your ideas & feature requests directly to our product requirements team! We look forward to hearing from you.

Add Brickhouse UDF for Hive Explode Multiple Arrays

We want to calculate value by using Exploding multiple arrays at the same time.

 

If we could use the following udf, it would be easy to write query.
https://github.com/klout/brickhouse
http://linuxandryan.wordpress.com/2014/01/17/hive-explode-multiple-arrays/
http://brickhouseconfessions.wordpress.com/2013/03/07/exploding-multiple-arrays-at-the-same-time-with-numeric_range/

 

posexplode is a workaround. But JOINi with a lot of key would be not good for a large table size. :

SELECT COOKIE ,
ePRODUCT_ID,
eCAT_ID,
eQTY
FROM TABLE 
LATERAL VIEW posexplode(PRODUCT_ID) ePRODUCT_IDAS seqp, ePRODUCT_ID
LATERAL VIEW posexplode(CAT_ID) eCAT_ID AS seqc, eCAT_ID
LATERAL VIEW posexplode(QTY) eQTY AS seqq, eDateReported
WHERE seqp = seqc AND seqc = seqq;
  • Toru Takahashi
  • Oct 18 2017