Scoring mobile-friendliness

[Fengtan]

In April 2015 Google rolled out their mobile-friendly update

which boosts the ranking of mobile-friendly pages on mobile search results.

In March 2016 they announced another update

that increases the effect of mobile-friendliness.

Does Nutch support mobile-friendliness scoring ? If not, then maybe that is
something we could implement ? For instance we could evaluate the
mobile-friendliness of web pages and index the resulting score into a new
field.

I have researched libraries that we could use and the only one I found is
the W3C's MobileOK Checker which provides an online assessment tool
 as well as recommendations
 and a reference
implementation of these recommendations in Java
. It looks like the project
was last updated in 2011.

Is this something we would be interested in ? Do we know about similar
libraries to evaluate the mobile-friendliness of pages ?

Re: master branch, solr indexer fails with a message that I don't understand

[kaveh minooie]

Hey Furkan

here is the issue: https://issues.apache.org/jira/browse/NUTCH-2267

On 05/23/2016 02:51 PM, kaveh minooie wrote:

I found this https://issues.apache.org/jira/browse/SOLR-7657

let me give it a shot. I'll create an issue if it doen't work. And if
you have a patch that needs testing, you can also email me that as well
( or github url, or ... ) .

On 05/23/2016 02:46 PM, Furkan KAMACI wrote:

Hi,

Could you create a Jira ticket for it? I can create a patch for you to
try?

Kind Regards,
Furkan KAMACI

On Tue, May 24, 2016 at 12:00 AM, kaveh minooie  wrote:


I am using master branch, solr is version 6 and in cloud
configuration. I
am using 'cloud' for solr.server.type in nutch-site.xml.

does this make sense to anybody?

16/05/23 13:52:27 INFO mapreduce.Job:  map 100% reduce 100%
16/05/23 13:52:27 INFO mapreduce.Job: Task Id :
attempt_1462499602101_0119_r_00_0, Status : FAILED
Error: Bad return type
Exception Details:
  Location:

org/apache/solr/client/solrj/impl/HttpClientUtil.createClient(Lorg/apache/solr/common/params/SolrParams;Lorg/apache/http/conn/ClientConnectionManager;)Lorg/apache/http/impl/client/CloseableHttpClient;

@58: areturn
  Reason:
Type 'org/apache/http/impl/client/DefaultHttpClient' (current frame,
stack[0]) is not assignable to
'org/apache/http/impl/client/CloseableHttpClient' (from method
signature)
  Current Frame:
bci: @58
flags: { }
locals: { 'org/apache/solr/common/params/SolrParams',
'org/apache/http/conn/ClientConnectionManager',
'org/apache/solr/common/params/ModifiableSolrParams',
'org/apache/http/impl/client/DefaultHttpClient' }
stack: { 'org/apache/http/impl/client/DefaultHttpClient' }
  Bytecode:
0x000: bb00 0359 2ab7 0004 4db2 0005 b900 0601
0x010: 0099 001e b200 05bb 0007 59b7 0008 1209
0x020: b600 0a2c b600 0bb6 000c b900 0d02 002b
0x030: b800 104e 2d2c b800 0f2d b0
  Stackmap Table:
append_frame(@47,Object[#143])

16/05/23 13:52:28 INFO mapreduce.Job:  map 100% reduce 0%



--
Kaveh Minooie







--
Kaveh Minooie

Re: master branch, solr indexer fails with a message that I don't understand

[kaveh minooie]

I found this https://issues.apache.org/jira/browse/SOLR-7657

let me give it a shot. I'll create an issue if it doen't work. And if 
you have a patch that needs testing, you can also email me that as well 
( or github url, or ... ) .


On 05/23/2016 02:46 PM, Furkan KAMACI wrote:

Hi,

Could you create a Jira ticket for it? I can create a patch for you to try?

Kind Regards,
Furkan KAMACI

On Tue, May 24, 2016 at 12:00 AM, kaveh minooie  wrote:


I am using master branch, solr is version 6 and in cloud configuration. I
am using 'cloud' for solr.server.type in nutch-site.xml.

does this make sense to anybody?

16/05/23 13:52:27 INFO mapreduce.Job:  map 100% reduce 100%
16/05/23 13:52:27 INFO mapreduce.Job: Task Id :
attempt_1462499602101_0119_r_00_0, Status : FAILED
Error: Bad return type
Exception Details:
  Location:

org/apache/solr/client/solrj/impl/HttpClientUtil.createClient(Lorg/apache/solr/common/params/SolrParams;Lorg/apache/http/conn/ClientConnectionManager;)Lorg/apache/http/impl/client/CloseableHttpClient;
@58: areturn
  Reason:
Type 'org/apache/http/impl/client/DefaultHttpClient' (current frame,
stack[0]) is not assignable to
'org/apache/http/impl/client/CloseableHttpClient' (from method signature)
  Current Frame:
bci: @58
flags: { }
locals: { 'org/apache/solr/common/params/SolrParams',
'org/apache/http/conn/ClientConnectionManager',
'org/apache/solr/common/params/ModifiableSolrParams',
'org/apache/http/impl/client/DefaultHttpClient' }
stack: { 'org/apache/http/impl/client/DefaultHttpClient' }
  Bytecode:
0x000: bb00 0359 2ab7 0004 4db2 0005 b900 0601
0x010: 0099 001e b200 05bb 0007 59b7 0008 1209
0x020: b600 0a2c b600 0bb6 000c b900 0d02 002b
0x030: b800 104e 2d2c b800 0f2d b0
  Stackmap Table:
append_frame(@47,Object[#143])

16/05/23 13:52:28 INFO mapreduce.Job:  map 100% reduce 0%



--
Kaveh Minooie





--
Kaveh Minooie

Re: master branch, solr indexer fails with a message that I don't understand

[Furkan KAMACI]

Hi,

Could you create a Jira ticket for it? I can create a patch for you to try?

Kind Regards,
Furkan KAMACI

On Tue, May 24, 2016 at 12:00 AM, kaveh minooie  wrote:

> I am using master branch, solr is version 6 and in cloud configuration. I
> am using 'cloud' for solr.server.type in nutch-site.xml.
>
> does this make sense to anybody?
>
> 16/05/23 13:52:27 INFO mapreduce.Job:  map 100% reduce 100%
> 16/05/23 13:52:27 INFO mapreduce.Job: Task Id :
> attempt_1462499602101_0119_r_00_0, Status : FAILED
> Error: Bad return type
> Exception Details:
>   Location:
>
> org/apache/solr/client/solrj/impl/HttpClientUtil.createClient(Lorg/apache/solr/common/params/SolrParams;Lorg/apache/http/conn/ClientConnectionManager;)Lorg/apache/http/impl/client/CloseableHttpClient;
> @58: areturn
>   Reason:
> Type 'org/apache/http/impl/client/DefaultHttpClient' (current frame,
> stack[0]) is not assignable to
> 'org/apache/http/impl/client/CloseableHttpClient' (from method signature)
>   Current Frame:
> bci: @58
> flags: { }
> locals: { 'org/apache/solr/common/params/SolrParams',
> 'org/apache/http/conn/ClientConnectionManager',
> 'org/apache/solr/common/params/ModifiableSolrParams',
> 'org/apache/http/impl/client/DefaultHttpClient' }
> stack: { 'org/apache/http/impl/client/DefaultHttpClient' }
>   Bytecode:
> 0x000: bb00 0359 2ab7 0004 4db2 0005 b900 0601
> 0x010: 0099 001e b200 05bb 0007 59b7 0008 1209
> 0x020: b600 0a2c b600 0bb6 000c b900 0d02 002b
> 0x030: b800 104e 2d2c b800 0f2d b0
>   Stackmap Table:
> append_frame(@47,Object[#143])
>
> 16/05/23 13:52:28 INFO mapreduce.Job:  map 100% reduce 0%
>
>
>
> --
> Kaveh Minooie
>

master branch, solr indexer fails with a message that I don't understand

[kaveh minooie]

I am using master branch, solr is version 6 and in cloud configuration. 
I am using 'cloud' for solr.server.type in nutch-site.xml.


does this make sense to anybody?

16/05/23 13:52:27 INFO mapreduce.Job:  map 100% reduce 100%
16/05/23 13:52:27 INFO mapreduce.Job: Task Id : 
attempt_1462499602101_0119_r_00_0, Status : FAILED

Error: Bad return type
Exception Details:
  Location:

org/apache/solr/client/solrj/impl/HttpClientUtil.createClient(Lorg/apache/solr/common/params/SolrParams;Lorg/apache/http/conn/ClientConnectionManager;)Lorg/apache/http/impl/client/CloseableHttpClient; 
@58: areturn

  Reason:
Type 'org/apache/http/impl/client/DefaultHttpClient' (current 
frame, stack[0]) is not assignable to 
'org/apache/http/impl/client/CloseableHttpClient' (from method signature)

  Current Frame:
bci: @58
flags: { }
locals: { 'org/apache/solr/common/params/SolrParams', 
'org/apache/http/conn/ClientConnectionManager', 
'org/apache/solr/common/params/ModifiableSolrParams', 
'org/apache/http/impl/client/DefaultHttpClient' }

stack: { 'org/apache/http/impl/client/DefaultHttpClient' }
  Bytecode:
0x000: bb00 0359 2ab7 0004 4db2 0005 b900 0601
0x010: 0099 001e b200 05bb 0007 59b7 0008 1209
0x020: b600 0a2c b600 0bb6 000c b900 0d02 002b
0x030: b800 104e 2d2c b800 0f2d b0
  Stackmap Table:
append_frame(@47,Object[#143])

16/05/23 13:52:28 INFO mapreduce.Job:  map 100% reduce 0%



--
Kaveh Minooie

Re: [ANNOUNCE] New Nutch committer and PMC - Karanjeet Singh

[Karanjeet Singh]

Hi Sebastian,

Thanks for the invitation and warm welcome.


Hello Everyone,

I am glad to be on board and having this opportunity to work with all of
you.

I am a graduate student at the University of Southern California (USC)
pursuing my Master’s in Computer Science. Prior to this, I was working as a
web developer at Computer Sciences Corporation (CSC), India.  At CSC, I
have developed applications for a global payments technology company
adhering to PCI DSS standards.

And now, I am starting my summer internship at NASA JPL.

Last year, in 2015, I took a course named Information Retrieval (IR) under
Prof. Chris Mattmann where I got the opportunity to learn and work on Nutch
1.x. This was the time when I started working on some of its bugs. The
semester ended but not the interest and therefore I moved ahead working on
Nutch plugins, particularly, HtmlUnit and Selenium.

During this summer, I plan to make more contributions and help the
community grow. Also, I plan to port Nutch backend on Spark for an improved
performance and better after-crawl analysis. I am also interested in
working on real-time crawl analysis in Nutch through a clean and easy to
understand visual interface.

I am excited to be a part of this community!!!
Regards,
Karanjeet Singh
USC


On Sun, May 22, 2016 at 12:51 PM, Sebastian Nagel <
wastl.na...@googlemail.com> wrote:

> Dear all,
>
> on behalf of the Nutch PMC it is my pleasure to announce
> that Karanjeet Singh has joined the Nutch team as committer
> and PMC member. Karanjeet, would you mind to introduce
> yourself and tell the Nutch community about your relation
> to Apache Nutch, what you have done or plan to do, etc.?
>
> Congratulations and welcome on board!
>
> Regards,
> Sebastian
>

ᐧ