Custom Map Reduce Program on Hive,whats the Rule?how about input and output?

⇧

⇩

42 views

Hello Fellow Developer,i got stuck a few day because i want to create custom map reduce program based on my query on hive,not few example i've found after googling and still make me confuse about the rule.

what is the rule to create my custom mapreduce program,how about the mapper and reducer class?

anyone can provide any solutions?

i want to develop this program in java,bust i still stuck,and then when formatting output in collector,how i format the result in mapper and reducer class?

anybody want to give me some example and explanation about this kind of stuff?

thanks for anyone attention and sorry for my bad english :D

hadoop mapreduce hive

asked May 30, 2011 11:24 am CDT

fahmi

posted via StackOverflow

0 Answers

Be the first to answer this question

Similar discussion threads

Updated: (HIVE-177) Allow specifying custom input and output format in CREATE TABLE
July 23, 2010 01:15:53 PM
[ https://issues.apache.org/jira/browse/HIVE-177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-177:

Custom Input/Output Format for Pig.
March 2, 2011 05:24:36 PM
Hi, I notice that Pig has its own Input/Output Format to talk to Hadoop. If I want to use my custom Input/Output Format and Reader and Writer, is there a way to hook them up with Pig? If so, how? Thanks, Jane

how to get handle to the output of Hive query and know the control flow of the program
January 19, 2011 12:42:43 PM
hi, Can someone please let me know how can i find out the control flow in Hive of a query that is executed ? I want to know the modules called so that I can modify them for my purpose. Actually, finally i want to have handle to the output of the…

Updated: (HIVE-37) Hive: new syntax for specifying custom map/reduce scripts
January 24, 2011 03:34:09 PM
[ https://issues.apache.org/jira/browse/HIVE-37?page=com.atlassian.jira. plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-37:

Updated: (HIVE-56) The reducer output is not created if the mapper input is empty
October 17, 2010 10:31:49 PM
[ https://issues.apache.org/jira/browse/HIVE-56?page=com.atlassian.jira. plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-56:

Directing output from Hive MR second custom MR job
August 25, 2010 04:13:56 AM
Hello, Any pointers about directing output from Hive MR as input for another (not hive based) MR job? I would like to use hive to select a subset of my data and then process that data using my custom code. I think the correct them is "map reduce…

Created: (HIVE-2004) Configuring a Map-Reduce job with dynamic input in case of a LIMIT query that d
February 24, 2011 04:15:01 PM
Configuring a Map-Reduce job with dynamic input in case of a LIMIT query that does not contain an ORDER BY

Created: (HIVE-1600) need to support hook input/output lists for test result determinism
August 26, 2010 04:55:18 PM
need to support hook input/output lists for test result determinism

FileNotFoundException in Reduce step when running importtsv program
March 17, 2011 01:59:08 PM
Hi all, I am attempting to bulk load data into HBase using the importtsv program. I have a very wide table (about 200 columns, 2 column families), and right now I'm trying to load in data from a single data file with 1 million rows. Importtsv…

Fwd: FileNotFoundException in Reduce step when running HBase importtsv program
March 17, 2011 03:26:17 PM
I sent this to the HBase mailing list, but thought I would also send this here in case anyone has any idea what might be going on. Thanks ---------- Forwarded message

"Reduce input groups" vs "Reduce input records"
March 25, 2011 11:25:15 AM
Hi, in this MR example, it exists the field "Reduce input groups" and "Reduce input records". What's the difference between these 2 fields? $ hadoop jar cloud9.jar edu.umd.cloud9.example.simple.DemoWordCount data/bible+shakes.nopunc wc 1…

Updated: (HIVE-256) map side aggregation : number of output rows is same as number of input rows
July 23, 2010 11:52:53 AM
[ https://issues.apache.org/jira/browse/HIVE-256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Carl Steinbach updated HIVE-256:

using output from one job as input to another
March 2, 2011 08:22:44 PM
Hi I am new to Hadoop, so maybe I am missing something obvious. I have written a small map reduce program that runs two jobs. I want the output of the first job to serve as the input to the second job. Here is what my driver code looks like: …

Created] (HIVE-2096) throw a error if the input is larger than a threshold for index input format
April 5, 2011 07:08:46 PM
throw a error if the input is larger than a threshold for index input format

reduce input is compressed?
February 15, 2011 09:12:15 AM
Hi, When the compression in on, the compressed map intermediate files are transfered to the reduce side as compressed data? Thanks,

A function as an input for map/reduce
May 3, 2011 04:56:08 PM
Hi, I watched "MapReducing Big Data With Riak and Luwak"[1] recorded webinar. And it is based on a possibility to use a function ("dynamic inputs" @ ~8:04 in the video) to actually generate inputs for the query. It raised two questions: 1. Is…

HBase as input AND output?
October 13, 2010 03:08:46 PM
Hi, I was wondering how I can query data stored in HBase and remembered Hive's HBase integration: http://wiki.apache.org/hadoop/Hive/HBaseIntegration After watching John Sichi's video …

HBase as input AND output
October 13, 2010 04:18:11 PM
Hi, I was wondering how I can query data stored in HBase and remembered Hive's HBase integration: http://wiki.apache.org/hadoop/Hive/HBaseIntegration After watching John Sichi's video …

RE: Applications creates bigger output than input?
April 29, 2011 09:52:18 AM
If you are looking for exponential output volumes then run a market basket test. You will need only a few thousand baskets with 100 items or so. Set min support very low (5%) and confidence at 80% and you should get a flood of data. Any FIS…

Applications creates bigger output than input?
April 29, 2011 07:03:35 AM
One of assumptions map reduce made, I think, is that size of map's output is smaller than input. Although we can see many applications have the same size of output with input, like, sort, merge,etc. For my benchmark purpose, I am looking for some…

Questions and Answers are licensed under cc-wiki license.

Custom Map Reduce Program on Hive,whats the Rule?how about input and output?

0 Answers

Be the first to answer this question

Join with account you already have