tag:blogger.com,1999:blog-457115006574868851.post9037045755757763433..comments2015-10-26T06:43:29.342-05:00Comments on Big Data: The New Hadoop API 0.20.xA. Soner Balkirhttp://www.blogger.com/profile/10156476894897813837noreply@blogger.comBlogger16125tag:blogger.com,1999:blog-457115006574868851.post-87689271725380703742012-11-30T11:27:11.968-06:002012-11-30T11:27:11.968-06:00Very helpful! Thank you.Very helpful! Thank you.Anonymoushttps://www.blogger.com/profile/00784093826682559452noreply@blogger.comtag:blogger.com,1999:blog-457115006574868851.post-83488074482445082802012-11-04T02:32:12.297-06:002012-11-04T02:32:12.297-06:00This is why many companies are now ordering their ... This is why many companies are now ordering their API gages from online sources where they get better support because everything can end up being done digitally. This makes it relatively simple for those who are trying to get the best products for the job they do.<br /><br /><a href="http://www.headhuntable.com/j/apis-jobs" rel="nofollow">apis jobs</a><br />alfredhttps://www.blogger.com/profile/01349478311612577696noreply@blogger.comtag:blogger.com,1999:blog-457115006574868851.post-30569986342822516322012-08-24T04:30:03.032-05:002012-08-24T04:30:03.032-05:00What is the replacement for TextInputFormat in the...What is the replacement for TextInputFormat in the new API?? When I am using it, I get a warning that its deprecated.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-457115006574868851.post-35628965939555518142012-06-14T03:31:19.527-05:002012-06-14T03:31:19.527-05:00cool much needed info thankscool much needed info thanksAnonymousnoreply@blogger.comtag:blogger.com,1999:blog-457115006574868851.post-81580817119384961232011-10-02T10:15:26.608-05:002011-10-02T10:15:26.608-05:00FYI: http://stackoverflow.com/questions/7626077/ma...FYI: http://stackoverflow.com/questions/7626077/mapreducebase-and-mapper-deprecated<br /><br />Thanks!Garyhttps://www.blogger.com/profile/05940167300305492314noreply@blogger.comtag:blogger.com,1999:blog-457115006574868851.post-71671829023477822912011-06-13T11:33:45.764-05:002011-06-13T11:33:45.764-05:00Thanks for the post. I have a specific question ab...Thanks for the post. I have a specific question about the new implementation.<br /><br />The class Mapper.Context is a nested class defined in Mapper. So where is this class instantiated and passed as a parameter to the map() method when an application is running? As far as I know, one cannot instantiate a nested class outside of the superclass. Thanks!Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-457115006574868851.post-25683112600809343492011-01-04T20:32:39.152-06:002011-01-04T20:32:39.152-06:00Job job = new Job(new Configuration());
Configurat...Job job = new Job(new Configuration());<br />Configuration conf = job.getConfiguration();<br />conf.set(name, value);<br /><br />Will this work ?Tusharnoreply@blogger.comtag:blogger.com,1999:blog-457115006574868851.post-44977390519450018332010-10-02T21:48:53.810-05:002010-10-02T21:48:53.810-05:00Configuration conf = new Configuration();
conf.set...Configuration conf = new Configuration();<br />conf.setInt("num2",Integer.parseInt(args[2]); <br />Job job = new Job(conf, "Size");<br /><br />I am able to set my variable num2 to a value from arguments.<br /><br />But how do I access it from Mapper class.<br /> <br />public static class MapClass extends Mapper {<br /><br /> private static int N;<br /> protected void setup(Context context)<br /> {<br /> N= context.getConfiguration().getInt("num2",1);<br /><br /> }<br /><br /> public void map(Object key, Text value, Context context) throws IOException, InterruptedException {<br /> <br /> //I use N here<br /><br /> }<br /> }<br /> }<br /><br />public int run(String[] args) throws Exception {<br /> Configuration conf = new Configuration();<br /> conf.setInt("num2",Integer.parseInt(args[3])); <br /> Job job = new Job(conf, "Size");<br /><br />///////<br />}gbaghttps://www.blogger.com/profile/12286940014294250924noreply@blogger.comtag:blogger.com,1999:blog-457115006574868851.post-46786285804708318222010-08-01T21:37:39.262-05:002010-08-01T21:37:39.262-05:00yep. I had that problem too. need to set the param...yep. I had that problem too. need to set the parameter before you instantiate the job. thanksThe Unionhttps://www.blogger.com/profile/10642929225950850535noreply@blogger.comtag:blogger.com,1999:blog-457115006574868851.post-64757363722306288082010-07-17T07:29:40.324-05:002010-07-17T07:29:40.324-05:00Using the old API, in Mapper.configure() with Job....Using the old API, in Mapper.configure() with Job.conf.get() I am able to get the configuration map.input.file (file being processed by the mapper).<br />When I tried to print the list of all configurations using the configurations iterator, I dont see this configuration any more. <br />Am I missing something?Jairam Chandarhttp://www.facebook.com/jairamcnoreply@blogger.comtag:blogger.com,1999:blog-457115006574868851.post-88874275874449266402010-07-01T19:18:31.227-05:002010-07-01T19:18:31.227-05:00Hello Soner,
I have been struggling...Hello Soner,<br /> I have been struggling with the same issue. I have serialized my object into string as follows:<br /><br />Configuration conf = new Configuration(); <br />JobConf conf1 = new JobConf(conf,new.class)<br />ByteArrayOutputStream baos = new ByteArrayOutputStream();<br /> ObjectOutputStream oos = new ObjectOutputStream(baos);<br /> oos.writeObject(gList); <br /> <br />String gridListString = new String(baos.toString());<br /> conf1.set("gridList", gridListString);<br /> oos.close();<br /><br />And now, in the mapper i am deserializing my string into object by overriding configure function as follows:<br />public void configure(JobConf job)<br />{<br />super.configure(job);<br />String mygridList = job.get("gridList");<br /><br />ByteArrayInputStream bios = new ByteArrayInputStream(getBytes(mygridList));<br /> <br />ObjectInputStream ois = new ObjectInputStream(bios);<br /> <br />gList = (GridDetails) (ois.readObject()); }<br />ois.close();<br /><br />} <br /><br />While i am trying to use the gList in the map function, it returns a NullException.<br /><br />Could you please help me in this issue. Any suggestions are valuable to me.<br /><br />Thanks,<br />SamAnonymoushttps://www.blogger.com/profile/17810848315102571438noreply@blogger.comtag:blogger.com,1999:blog-457115006574868851.post-13444029770619339992010-07-01T18:47:49.619-05:002010-07-01T18:47:49.619-05:00Hello Soner,
I have been struggling...Hello Soner,<br /> I have been struggling with the same issue. I have serialized my object into string as follows:<br /><br />Configuration conf = new Configuration(); <br />JobConf conf1 = new JobConf(conf,new.class)<br />ByteArrayOutputStream baos = new ByteArrayOutputStream();<br /> ObjectOutputStream oos = new ObjectOutputStream(baos);<br /> oos.writeObject(gList); <br /> <br />String gridListString = new String(baos.toString());<br /> conf1.set("gridList", gridListString);<br /> oos.close();<br /><br />And now, in the mapper i am deserializing my string into object by overriding configure function as follows:<br />public void configure(JobConf job)<br />{<br />super.configure(job);<br />String mygridList = job.get("gridList");<br /><br />ByteArrayInputStream bios = new ByteArrayInputStream(getBytes(mygridList));<br /> <br />ObjectInputStream ois = new ObjectInputStream(bios);<br /> <br />gList = (GridDetails) (ois.readObject()); }<br />ois.close();<br /><br />} <br /><br />While i am trying to use the gList in the map function, it returns a NullException.<br /><br />Could you please help me in this issue. Any suggestions are valuable to me.<br /><br />Thanks,<br />SamAnonymousnoreply@blogger.comtag:blogger.com,1999:blog-457115006574868851.post-80553776484547764772010-04-23T13:04:03.960-05:002010-04-23T13:04:03.960-05:00thanksthanksAnonymousnoreply@blogger.comtag:blogger.com,1999:blog-457115006574868851.post-20945785045071771052010-03-22T18:38:18.985-05:002010-03-22T18:38:18.985-05:00I'll have to look at that, thanks for the hint...I'll have to look at that, thanks for the hint.Keithhttp://keithwiley.comnoreply@blogger.comtag:blogger.com,1999:blog-457115006574868851.post-945252326813415292010-03-10T11:02:45.569-06:002010-03-10T11:02:45.569-06:00Keith,
I think you may be doing something wrong w...Keith,<br /><br />I think you may be doing something wrong with the order of setting up the parameters. You must set the configuration parameters before creating the Job object. For instance, if you have something like:<br /><br />conf.set("parameter1", "value1");<br />Job job = new Job(conf);<br />conf.set("parameter2", "value2");<br /><br />you won't be able to get the value of "parameter2" from the context objects, it'll be null. Does your code have something similar to above?A. Soner Balkirhttps://www.blogger.com/profile/10156476894897813837noreply@blogger.comtag:blogger.com,1999:blog-457115006574868851.post-63919885897063079162010-03-04T19:29:30.883-06:002010-03-04T19:29:30.883-06:00Thanks for posting this. I'm having a problem...Thanks for posting this. I'm having a problem though.<br /><br />I used to pass key/value data into the mapper using JobConf.set() and setStrings() and retrieving them in Mapper.configure() with Job.conf.get() and getStrings().<br /><br />I have replaced the key/value insertion with Configuration.set.<br /><br />Mapper.configure(JobConf) seems to have been replaced with Mapper.setup(Context), so in setup() I call Context.getConfiguration().get().<br /><br />...but the key isn't there! In fact, if I get the configurations iterator and print out all the keys, it is definitely not there, at all.<br /><br />Any help appreciated. Thanks.kwiley@keithwiley.comhttp://keithwiley.comnoreply@blogger.com