bq. Caused by: java.lang.NullPointerException Can you show the remaining stack trace ?
Thanks On Tue, Mar 22, 2016 at 5:43 AM, Hafsa Asif <[email protected]> wrote: > Hello everyone, > I am trying to get benefits of DataFrames (to perform all SQL BASED > operations like 'Where Clause', Joining etc.) as mentioned in > > https://spark.apache.org/docs/1.5.1/api/java/org/apache/spark/sql/DataFrame.html > . > > I am using, Aerospike and Spark (1.4.1) Java Client in Spring Framework . > My scenario as below: > (My apparent guide is : > > http://www.programcreek.com/java-api-examples/index.php?source_dir=deep-examples-master/deep-aerospike/ > ) > > 1. I have a DeepSparkContext. > 2. Preparing DataFrame from Aerospike in this way: > > public DataFrame createDataFrame (String setName, String[] outputColumns, > String userId ){ > > DataFrame frame = null; > > ExtractorConfig setCells = new ExtractorConfig<>(Cells.class); > setCells.putValue(ExtractorConstants.HOST, > configurationHandler.aerospikeHost()) > .putValue(ExtractorConstants.PORT, > configurationHandler.aerospikePort()) > .putValue(ExtractorConstants.NAMESPACE, > configurationHandler.getAerospikeNamespace()) > .putValue(ExtractorConstants.SET, setName) > .putValue(ExtractorConstants.INPUT_COLUMNS, outputColumns) > > > setCells.setExtractorImplClass(AerospikeCellExtractor.class); > > try { > > frame= deepSparkContext.createJavaSchemaRDD(setCells); > } catch (UnsupportedDataTypeException undt){ > log.error(undt.getMessage()); > } > > return frame; > } > 3. In another method, I am processing DataFrame in this way: > public void getActivePush (String userId, Integer status){ > > String[] activePushColumns = new String[] {"clientId", "userId", > "status"}; > DataFrame activePushFrame = > createDataFrame(configurationHandler.getAerospikePushActivationSet(), > activePushColumns, userId); > activePushFrame.registerTempTable("activePush"); > DataFrame messagesFiltered = deepSparkContext.sql("SELECT * FROM > activePush"); > log.debug("first : " + messagesFiltered.first()); > } > > The above method gives successfully the first element. > 4. But, If I use : > DataFrame messagesFiltered = deepSparkContext.sql("SELECT * FROM activePush > WHERE userId = 'user1'"); > It gives me the following exception: > [2016-03-22 13:40:15.413] boot - 10493 ERROR [main] --- SpringApplication: > Application startup failed > org.springframework.beans.factory.BeanCreationException: Error creating > bean > with name 'MyAnalyzer': Invocation of init method failed; nested exception > is java.lang.NullPointerException > at > > org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.postProcessBeforeInitialization(InitDestroyAnnotationBeanPostProcessor.java:136) > at > > org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyBeanPostProcessorsBeforeInitialization(AbstractAutowireCapableBeanFactory.java:408) > at > > org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1564) > at > > org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:539) > at > > org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:476) > at > > org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:303) > at > > org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230) > at > > org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:299) > at > > org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:194) > at > > org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:755) > at > > org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:762) > at > > org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:480) > at > > org.springframework.boot.SpringApplication.refresh(SpringApplication.java:690) > at > org.springframework.boot.SpringApplication.run(SpringApplication.java:322) > > springframework.boot.SpringApplication.run(SpringApplication.java:970) > at > org.springframework.boot.SpringApplication.run(SpringApplication.java:959) > at > > com.matchinguu.analytics.AnalyticalEngineApplication.main(AnalyticalEngineApplication.java:14) > Caused by: java.lang.NullPointerException > > Please guide me that how to apply where clause conditions/filters in > DataFrame? I have also a plan to create multiple DataFrames and to apply > JOIN operation between them, it will be good if u give me some good > examples > about it. > > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/Issue-wihle-applying-filters-conditions-in-DataFrame-in-Spark-tp26560.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > >
