CFPs and the lists

[Hugh Glaser]

Hmmm.
So I am enjoying the new regime without CfPs on the LOD list (many thanks, 
Phil!).
However, I now find myself thinking I will unsubscribe from the SemWeb list, 
since it is almost all CfPs, few, if any, of which I want.

I think this may be an unintended consequence (although probably predictable) - 
losing people from SemWeb.

There isn’t really a question here - I just thought I would report it.
If there is a question: dare I suggest that now things have settled down, and 
we can see how things are working, that we might want to revisit the idea of 
having a separate list for CfPs, and reclaim the SemWeb list for discussion? 
(Sorry Phil?)
Or is the answer that I simply set mail filters and carry on?

Best
Hugh

Re: Where are the Linked Data Driven Smart Agents (Bots) ?

[Juan Sequeda]

On Thu, Jul 7, 2016 at 12:51 PM, Ruben Verborgh 
wrote:

> Hi Juan,
>
> Seems like we mostly agree—short remarks below.
>
> > One thing is science. Another is engineering.
>
> Perhaps we need Semantic Web Engineering conferences then as well!
>

That's why you have developer conferences, etc. ISWC this year had a
Applications track. And it doesn't have to be semantic web specific.


>
> > If we don't know the right evaluation metrics (I agree with you that we
> don't), then that is the current challenge we, as a semantic web scientific
> community, have to tackle.
>
> Indeed, but I've found the scientific community to be not so open to new
> evaluation metrics either. There is insufficient agreement on (and too
> limited knowledge of) the right scientific methodology to tackle such novel
> problems.
>

Well, I don't know what you have proposed. But it seems that you haven't
made a convincing argument :P (if you want to discuss offline what you are
doing, send me an email)


>
> > It shouldn't discourage you... on the contrary, it should encourage you
> to identify novel ways to evaluate what you are doing and convince the
> community why it is important.
>
> The trouble is you don't have to convince the entire community (with whom
> you can have an open dialog), but a tiny set of anonymous reviewers (for
> whom the known paths are often easier to judge).
>

Paper reviewing... that is different topic I don't want to get into here.


> My remark was precisely that convincing is hard once you move away from
> the known paths.
>
> So the scientific community, which is a large part of the total Semantic
> Web community, might in that sense be hampering real novelty—from science
> and engineering alike, whichever might be the difference.
>

There is a clear difference between Science and Engineering. That is my
point.  Science is about understanding what is unknown. In this case, it is
not well known how to evaluate new types of systems. That is what needs to
be studied. We need to figure out how to evaluate and evaluation. It is a
bit meta. Jim Hendler stated this 7 years ago and to the best of my
knowledge, this is an area that hasn't been tackled (PhD thesis anybody?)

If your goal is to get something out there and for it to be used, then why
do you bother spending time publishing papers. Look at all the open source
projects changing the world, with very little to no scientific
publications.

Just do what makes you happy and be the best at it. Strive for excellence!
I know you are :)


> Best,
>
> Ruben

Re: Where are the Linked Data Driven Smart Agents (Bots) ?

[Juan Sequeda]

Ruben,

One thing is science. Another is engineering.

Part of the scientific process is defining an experiment and doing the
evaluation. If we don't know the right evaluation metrics (I agree with you
that we don't), then that is the current challenge we, as a semantic web
scientific community, have to tackle. It shouldn't discourage you... on the
contrary, it should encourage you to identify novel ways to evaluate what
you are doing and convince the community why it is important.

In my opinion, evaluation of systems on the web is different that what the
CS community has been used to (CS is a young science). Evaluation has been
on run time, space consumed, precision, recall, sound, completeness. On the
web, those may not be the aspects we want to measures. In the words of Jim
Hendler:

“You want a good thesis? IR is based on precision and recall and the minute
you add semantics, it is a meaningless feature. Logic is based on soundness
and completeness. We don’t want soundness and completeness. We want a few
good answers quickly.”

– Prof. James A. Hendler, 2009, on the topic of answering queries over the
Semantic Web.
https://www.youtube.com/watch?v=sbmMxzOeZ-4

Here is a short position paper that Olaf Hartig and I wrote 5 years ago
(time flies when you are having fun): Towards a Query Language for the Web
of Data (A Vision Paper). http://arxiv.org/abs/1110.3017

My 2cts



--
Juan Sequeda, Ph.D
+1-575-SEQ-UEDA
www.juansequeda.com

On Wed, Jul 6, 2016 at 11:38 AM, Ruben Verborgh 
wrote:

> Hi,
>
> This is a very important question for our community,
> given that smart agents once were an important theme.
> Actually, the main difference we could bring with the SemWeb
> is that our clients could be decentralized
> and actually run on the client side, in contrast to others.
>
> One of the main problems I see is how our community
> (now particularly thinking about the scientific subgroup)
> receives submissions of novel work.
> We have evolved into an extremely quantitative-oriented view,
> where anything that can be measured with numbers
> is largely favored over anything that cannot.
>
> Given that the smart agents / bots field is quite new,
> we don't know the right evaluation metrics yet.
> As such, it is hard to publish a paper on this
> at any of the main venues (ISWC / ESWC / …).
> This discourages working on such themes.
>
> Hence, I see much talent and time going to
> incremental research, which is easy to evaluate well,
> but not necessarily as ground-breaking.
> More than a decade of SemWeb research
> has mostly brought us intelligent servers,
> but not yet the intelligent clients we wanted.
>
> So perhaps we should phrase the question more broadly:
> how can we as a community be more open
> to novel and disruptive technologies?
>
> Best,
>
> Ruben
>

Re: Where are the Linked Data Driven Smart Agents (Bots) ?

[Ruben Verborgh]

HI Krzysztof,

> this is all about finding the right balance

Definitely—but I have the feeling the balance
is currently tipped very much to one side
(and perhaps not the side that delivers
the most urgent components for the SemWeb).

> as we also do not want to have tons of 'ideas' 
> papers without any substantial content or proof of concept

Mere ideas would indeed not be sufficient;
but even papers with substantial content
and/or a proof of concept will have a difficult time
getting accepted if there is no evaluation
that satisfies the reviewers.
(And, lacking a framework to evaluate evaluations,
I see people typically choosing for things they know,
hence why incremental research gets accepted easily.)

Best,

Ruben

Re: Where are the Linked Data Driven Smart Agents (Bots) ?

[Krzysztof Janowicz]

As such, it is hard to publish a paper on this
at any of the main venues (ISWC / ESWC / …).
This discourages working on such themes.

Hence, I see much talent and time going to
incremental research, which is easy to evaluate well,
but not necessarily as ground-breaking.


Yes! I could not agree more. On the other hand, this is all about 
finding the right balance as we also do not want to have tons of 'ideas' 
papers without any substantial content or proof of concept. I remember 
that there was an ISWC session some years ago that tried to introduce 
such a 'bold ideas' track.


Krzysztof

On 07/06/2016 09:38 AM, Ruben Verborgh wrote:

Hi,

This is a very important question for our community,
given that smart agents once were an important theme.
Actually, the main difference we could bring with the SemWeb
is that our clients could be decentralized
and actually run on the client side, in contrast to others.

One of the main problems I see is how our community
(now particularly thinking about the scientific subgroup)
receives submissions of novel work.
We have evolved into an extremely quantitative-oriented view,
where anything that can be measured with numbers
is largely favored over anything that cannot.

Given that the smart agents / bots field is quite new,
we don't know the right evaluation metrics yet.
As such, it is hard to publish a paper on this
at any of the main venues (ISWC / ESWC / …).
This discourages working on such themes.

Hence, I see much talent and time going to
incremental research, which is easy to evaluate well,
but not necessarily as ground-breaking.
More than a decade of SemWeb research
has mostly brought us intelligent servers,
but not yet the intelligent clients we wanted.

So perhaps we should phrase the question more broadly:
how can we as a community be more open
to novel and disruptive technologies?

Best,

Ruben



--
Krzysztof Janowicz

Geography Department, University of California, Santa Barbara
4830 Ellison Hall, Santa Barbara, CA 93106-4060

Email: j...@geog.ucsb.edu
Webpage: http://geog.ucsb.edu/~jano/
Semantic Web Journal: http://www.semantic-web-journal.net