Friday, August 4, 2017

Pivot columns to rows in Apache Pig

Pivot columns to rows in Pig (Apache pig)
==============================

Transposing columns to rows in pig

Data input
id1, col1,col2,col3
id2,col1,col2,col3

Expected o/p
id1 col1 col3
id1 col2 col3
id2 col1 col3
id2 col2 col3


A = LOAD '/home/hadoop/work/surjan/pivotData.txt' using PigStorage(',','-noschema') as (ID: chararray, col2: chararray, col3: chararray,col4:chararray);
B = foreach A generate ID,CONCAT(CONCAT(col2,'#'),col3),col4;
C = foreach B generate ID, FLATTEN(TOKENIZE($1,'#')),col4;
dump C;