Re: [C3] Sitemap implemented in Scala
On 05.03.2009 00:33, Grzegorz Kossakowski wrote: Now I'll give a few words of comments why I find my implementation easier to follow. First of all, sitemap processing is divided into a few distinct stages: 1. Sitemap parsing 2. Sitemap reduction 3. Sitemap invocation 3.1 Actions execution 3.2 Pipeline building 3.3 Pipeline execution 3.4 Catching possible exception 4. If exception catched, reduction of handle errors node 5. Handling exception with reduction result What's more, for most of the time code is based on immutable structures which is the biggest advantage. What I found hard with current implementation hard is InvocationImpl passed around that quite a lot of state. If you combine it with InvocationResult switches (cases) that every node has to return in its invoke() method the result isn't really appealing. This sort of resembles mistakes from C2.x where big, mutable objects were passed around leading to unmaintainable code. I'm interested in hearing your opinions on that subject. I know that's rather hard to judge rather big piece of code in unfamiliar language but I'm sure you can easily get some overall feeling about my implementation and ideas behind it. I don't have the time to actually look into your implementation. I'm just wondering if all the insight views couldn't be used to improve the Java implementation rather than starting something in a different language. Joerg
Re: [C3] Sitemap implemented in Scala
Joerg Heinicke pisze:
I don't have the time to actually look into your implementation. I'm
just wondering if all the insight views couldn't be used to improve the
Java implementation rather than starting something in a different language.
It could.
What I'm doing at the moment is sort of wild experimentation that I don't know
where will lead me to. I'm reimplementing
sitemap in Scala because I thought that sitemap processing is something
complicated enough to get a real feel of
programming in Scala.
It's true that anything I write in Scala could be emulated in Java but when I
think about it I feel rather overwhelmed
by the possible verbosity of Java counterparts to Scala constructs. In my
implementation I exploit heavily strong typing
by defining lots of types and using lots of generics. I use closures and
pattern matching quite a lot.
Compare (Scala):
private def filterUnprefixedSimpleAttributes(attributes : MetaData) :
Map[String, String] =
Map() ++ attributes.flatMap(attr = attr match {
case ua : UnprefixedAttribute = List(ua.key -
ua.value.first.toString)
case _ = Nil
})
to translation of it (Java):
private MapString, String filterUnprefixedSimpleAttributes(MetaData
attributes) = {
return (new MapString, String()).putAll(attributes.flatMap(
new Function1MetaData, ListTuple2String, String() {
ListTuple2String, String apply(MetaData x) {
if (x istanceof UnprefixedAttribute) {
UnprefixedAttribute ua = (UnprefixedAttribute) x;
return new ListTuple2String, String().add(
new Tuple2String, String(ua.key(),
ua.value().first().toString()));
} else {
return new ListTuple2String, String();
}
})
}
And no, I didn't take the most complicated example from my code because I don't
it find funny to write such a spaghetti
code.
So certainly I could write the same in Java but I honestly believe that in Java
operating on strongly typed, immutable
collections is NOT fun and that's the reason nobody seems to do.
And if you skim through my code you'll find that it's more or less only about
mapping, reducing and filtering strongly
typed collections.
Still example above does not prove anything general (like Java sucks, let's do
it in Scala) but my reimplementation of
sitemap processing should provide a nice background for a discussion what we
want to see in C3.
For example, I'm interested in answering questions like if we are going to
introduce any other language apart from Java
in C3. Traditionally Cocoon had other languages integrated for years
(Javascript comes to mind) so it's rather natural
to ask this question. If answer was positive (based on some evaluation) another
question that pops up is why it can't be
Scala?
WARNING: Here follows some thoughts on personal experience with Scala that you
my safely ignore as they are rather
off-topic.
Might be that I'm extrapolating but my sitemap reimplementation in Scala did
work just after I finished all necessary
bits. I mean: I have written all the processing, integrated it into servlet so
it could work with the rest of C3 and it
did work just immediately. I didn't have any test-cases and I didn't run my
code while writing it.
After extensive testing with all samples C3 provides I found exactly one bug[1]
in that code. Of course, this does not
mean I don't have any other bugs but the probability that I have many of them
is really small.
Does Scala make me magically not making any mistakes? Certainly not. It only
informs me about them at the time of
writing code by reporting compilation errors.
This experience convinced me to have following standpoint:
If we are going to introduce any secondary language in Cocoon,
I would really prefer if it was strongly typed one.
Oh, and if you wonder how on the hell I could be typing non-trivial stuff for
more than one day without introducing
almost any bugs. It might be because my code cotains 5 if statements, no
return, break or continue statements and only a
few raw loops (like while, for or repeat). I believe that's the reason...
And yes, I have to admit that sitemap processing is somehow special case where
the power of concepts I've used simply
shines.
[1]
http://github.com/gkossakowski/apache-cocoon3/commit/845f7b61861827a17fb017625eb7f2281ac458df
--
Best regards,
Grzegorz Kossakowski
Re: [C3] Sitemap implemented in Scala
Grzegorz Kossakowski wrote:
Grzegorz Kossakowski pisze:
As Andreas pointed out, there is nothing wrong here but still my
repo without master branch was a little bit weird. I've pushed it
so now it clones without any warnings.
Still the best way to checkout is to create local branch on top of
my branch, as suggested by Andreas.
...
Thanks for interest in my work.
Just pinging. Reinhard, do you have any success/failure with my work?
I would like to receive some feedback as I would like to continue
with my work and prepare something more polished for our meeting at
ApacheCon EU in Amsterdam.
Sorry, forgot to answer. I have to admit that my Scala knowledge is far
from being depp enough to understand what your code does. But I've
started to learn Scala so that I can follow your explanations next week ;-)
--
Reinhard Pötz Managing Director, {Indoqa} GmbH
http://www.indoqa.com/en/people/reinhard.poetz/
Member of the Apache Software Foundation
Apache Cocoon Committer, PMC member reinh...@apache.org
Re: [C3] Sitemap implemented in Scala
Reinhard Pötz pisze: Sorry, forgot to answer. I have to admit that my Scala knowledge is far from being depp enough to understand what your code does. But I've started to learn Scala so that I can follow your explanations next week ;-) Ah, it's ok. :-) Personally, I find Scala simpler language than Java we all know but still it takes some time to dive into it. I recommend following materials: http://www.slideshare.net/jboner/pragmatic-real-world-scala-45-min-presentation - simply good http://www.slideshare.net/Odersky/fosdem-2009-1013261 - addressing various doubts and covers Scala - Java interoperability and other comparisons http://www.parleys.com/display/PARLEYS/Home#slide=1;title=Scala;talk=10485769 - a it's a great chance to listen to Martin Odersky, Scala creator. Interesting content of course. -- Best regards, Grzegorz Kossakowski
Re: [C3] Sitemap implemented in Scala
Grzegorz Kossakowski wrote: Reinhard Pötz pisze: Sorry, forgot to answer. I have to admit that my Scala knowledge is far from being depp enough to understand what your code does. But I've started to learn Scala so that I can follow your explanations next week ;-) Ah, it's ok. :-) Personally, I find Scala simpler language than Java we all know but still it takes some time to dive into it. I recommend following materials: http://www.slideshare.net/jboner/pragmatic-real-world-scala-45-min-presentation - simply good http://www.slideshare.net/Odersky/fosdem-2009-1013261 - addressing various doubts and covers Scala - Java interoperability and other comparisons http://www.parleys.com/display/PARLEYS/Home#slide=1;title=Scala;talk=10485769 - a it's a great chance to listen to Martin Odersky, Scala creator. Interesting content of course. Is there any decent IDE support for Scala? I am not even dreaming of scala refactorings. -- Leszek Gawron http://www.mobilebox.pl/krs.html CTO at MobileBox Ltd.
Re: [C3] Sitemap implemented in Scala
Leszek Gawron pisze: Is there any decent IDE support for Scala? I am not even dreaming of scala refactorings. I know someone will ask this question eventually. ;-) IDE support is probably the weakest of Scala. At the moment, I'm using snapshot of trunk version of Eclipse plug-in and some basic functionality works most of the time. What works: * code highlighting * code completion * basic refactoring (like rename but not in every case) * debugging * basic tool-tips displayed when hover on some element (e.g. method call) * errors reporting * code navigation using mouse and links in the code What doesn't work or is annoying: * method call parameters hints (rather annoying) * JUnit integration in Eclipse with test-cases written in Scala (Maven executes them just fine, though) * Bugs, bugs, bugs. It's rather often situation that Scala editor gets mad and does not highlight your code or code completion does not work. Anyway, what I find rather reassuring is the fact that just recently (if I'm not mistaken) Eclipse plug-in got new maintainer and some momentum. There is heavy refactoring happening in trunk and people responsible for that seem to really know how to develop Eclipse plug-ins. There is excellent feedback to my bug reports and most of them are fixed almost daily. This holds some promise for a bright future of this plug-in. I've heard that Netbeans plug-in offers little bit less of functionality than Eclipse one but is much more stable. I have tried it myself yet, though. IntelliJ seems to support Scala quite seriously but I haven't tried it myself as well. To sum it up: I find IDE support rather weak if you compare it to Eclipse's JDT for example but languges itself outweighs inconveniences. -- Best regards, Grzegorz Kossakowski
Re: [C3] Sitemap implemented in Scala
Grzegorz Kossakowski pisze: The last step of sitemap processing is invoking the result of the reduction. It's SitemapInvoker[5] class that is responsible for that. This class is only partly implemented as I didn't have time to deal with component factories needed to build actions and pipeline for execution. Anyway, this class extracts two distinct sets of nodes that we are interested in: * actions * pipeline components Actually this list lacks one item: * controllers (represeted by either call or redirect-to node) From this point, building a pipeline and executing actions is rather trivial task. I've refactored my code a little bit so it handles controllers and implemented sitemap building/execution. The implementation is in SitemapInvoker.invokeSitemap but is rather messy. It uses Invocation for actual pipeline building and execution. Messy or not, this code somehow works. You can checkout scalaSitemapEval: http://github.com/gkossakowski/apache-cocoon3/tree/scalaSitemapEval and compile it. Then just go and run samples. Some of them will work just fine. Some not, I have to check why they don't work but I don't have a time right now. One thing doesn't work completely: error handling. This is due to fact that Scala implementation uses completely different logic so if we want to have error handlers working there is no chance to reuse Invocation class. This is not a problem as I already plan to reimplement that part as well to make the whole example complete. Apart from SitemapInvoker.invokeSitemap method I think this code is rather polished. It lacks some comments, though. I plan to add some in upcomming days. Any feedback is greatly appreciated. -- Best regards, Grzegorz Kossakowski
Re: [C3] Sitemap implemented in Scala
Grzegorz Kossakowski pisze: As Andreas pointed out, there is nothing wrong here but still my repo without master branch was a little bit weird. I've pushed it so now it clones without any warnings. Still the best way to checkout is to create local branch on top of my branch, as suggested by Andreas. ... Thanks for interest in my work. Just pinging. Reinhard, do you have any success/failure with my work? I would like to receive some feedback as I would like to continue with my work and prepare something more polished for our meeting at ApacheCon EU in Amsterdam. -- Best regards, Grzegorz Kossakowski
Re: [C3] Sitemap implemented in Scala
Grzegorz Kossakowski wrote:
So here it comes:
http://github.com/gkossakowski/apache-cocoon3/tree/scalaSitemapEval
It's a branch out of our existing trunk found in svn. The only affected
module is cocoon-sitemap.
I tried to clone the repository by using following command:
git clone git://github.com/gkossakowski/apache-cocoon3.git
and got this outpout:
Initialized empty Git repository in [some-path]/.git/
remote: Counting objects: 7860, done.
remote: Compressing objects: 100% (2229/2229), done.
remote: Total 7860 (delta 3194), reused 7699 (delta 3071)
Receiving objects: 100% (7860/7860), 963.25 KiB | 187 KiB/s, done.
Resolving deltas: 100% (3194/3194), done.
warning: remote HEAD refers to nonexistent ref, unable to checkout.
I use git (msysgit 1.6.1-preview20081227) on Windows Vista.
Any suggestions (except changing the OS ;-) )?
--
Reinhard Pötz Managing Director, {Indoqa} GmbH
http://www.indoqa.com/en/people/reinhard.poetz/
Member of the Apache Software Foundation
Apache Cocoon Committer, PMC member reinh...@apache.org
Re: [C3] Sitemap implemented in Scala
On Saturday 07 March 2009 10:18:58 Reinhard Pötz wrote: Grzegorz Kossakowski wrote: So here it comes: http://github.com/gkossakowski/apache-cocoon3/tree/scalaSitemapEval It's a branch out of our existing trunk found in svn. The only affected module is cocoon-sitemap. I tried to clone the repository by using following command: git clone git://github.com/gkossakowski/apache-cocoon3.git and got this outpout: Initialized empty Git repository in [some-path]/.git/ remote: Counting objects: 7860, done. remote: Compressing objects: 100% (2229/2229), done. remote: Total 7860 (delta 3194), reused 7699 (delta 3071) Receiving objects: 100% (7860/7860), 963.25 KiB | 187 KiB/s, done. Resolving deltas: 100% (3194/3194), done. warning: remote HEAD refers to nonexistent ref, unable to checkout. I use git (msysgit 1.6.1-preview20081227) on Windows Vista. Any suggestions (except changing the OS ;-) )? This has nothing to do with your OS (although I'm still the opinion that changing to linux won't be the worst decision :P) The problem is that the HEAD is pointing to a master which does not exist. Doing git branch -a showing you all branches in the git repository. For me the output of the command was: [pie...@coprime apache-cocoon3]$ git branch -a origin/scalaPipeline origin/scalaSitemapEval As you can see no local branch exists. This could be created with git branch - b master origin/scalaPipeline [pie...@coprime apache-cocoon3]$ git checkout -b master origin/scalaPipeline warning: You appear to be on a branch yet to be born. warning: Forcing checkout of origin/scalaPipeline. Branch master set up to track remote branch refs/remotes/origin/scalaPipeline. Already on master by the way with this command git checkout -b otherBranch origin/scalaSitemapEval you can access the other branch in the repo. With git checkout BRANCHNAME you can switch between the branches. All localbranches could be seen with the command git branch Hope this helps! Andreas
Re: [C3] Sitemap implemented in Scala
Reinhard Pötz pisze: Grzegorz Kossakowski wrote: So here it comes: http://github.com/gkossakowski/apache-cocoon3/tree/scalaSitemapEval It's a branch out of our existing trunk found in svn. The only affected module is cocoon-sitemap. I tried to clone the repository by using following command: git clone git://github.com/gkossakowski/apache-cocoon3.git and got this outpout: Initialized empty Git repository in [some-path]/.git/ remote: Counting objects: 7860, done. remote: Compressing objects: 100% (2229/2229), done. remote: Total 7860 (delta 3194), reused 7699 (delta 3071) Receiving objects: 100% (7860/7860), 963.25 KiB | 187 KiB/s, done. Resolving deltas: 100% (3194/3194), done. warning: remote HEAD refers to nonexistent ref, unable to checkout. I use git (msysgit 1.6.1-preview20081227) on Windows Vista. As Andreas pointed out, there is nothing wrong here but still my repo without master branch was a little bit weird. I've pushed it so now it clones without any warnings. Still the best way to checkout is to create local branch on top of my branch, as suggested by Andreas. Any suggestions (except changing the OS ;-) )? It's not necessary because: 1. Vista *with SP1* is usable OS and after some tweaking is a nice OS. This is a great improvement on initial Vista release. 2. Git on Windows is much better than it used to be one year ago. There are even plugins like TortoiseGit (http://code.google.com/p/tortoisegit/) evolving very rapidly. The world is better than it was one year ago. ;-) Thanks for interest in my work. -- Best regards, Grzegorz Kossakowski
[C3] Sitemap implemented in Scala
Hi, I've just finished first phase of my work to implement sitemap in Scala. There were couple reasons why I started this work: 1. I wasn't satisfied with current implementation that seemed to me rather hard to follow. Still it looks much better than what we have in 2.2. 2. I wanted to try Scala-Java integration in real world scenario. 3. I wanted to find out if Scala code can be *easily* integrated into existing code-base 4. I had an idea how to implement sitemap processing in quite different way and I wanted to find out what I'll come up with eventually. So here it comes: http://github.com/gkossakowski/apache-cocoon3/tree/scalaSitemapEval It's a branch out of our existing trunk found in svn. The only affected module is cocoon-sitemap. Now I'll describe briefly why my implementation is different. Apart from obvious thing that I have used different language (Scala instead of Java) there are some more interesting things about it. The first thing that is rather apparent to anyone looking at this implementation that it's not OO-based. I don't want to go into details why I think OO does not make much sense for sitemap processing right now. This implies that (at least in current form) sitemap language cannot be extended in a way one can do in original C3's implementation. Anyway, I find this functionality barely useful so I didn't bother myself to make it working with my code even if I think it would be possible. All classes representing sitemap nodes are simple case classes defined in SitemapNode.scala[1]. Think of case classes as java beans on steroids grouped into set of distinct and well defined cases. The class responsible for parsing sitemap xml and producing it's representation is SitemapBuilder[2]. I suggest to you to skim over the code as there are couple of interesting things involved like pattern matching and xml literals. Mixing XML with the code may resemble XSP but I believe this is really a different case. Next step of sitemap processing is reduction. Sitemap tree reduction is defined as traversing the tree and erasing all conditional nodes like map:match and map:select. The result of reduction is a list of components (either actions or pipeline components) that were contained in conditional nodes that have their condition satisfied. Every component is accompanied by list of error handler applicable to the scope that given component is defined in. The class SitemapReductor[3] is responsible for sitemap reduction. This class shows[4] how one can easily and almost seamlessly call Java code from Scala. The only glitch is that match is keyword in scala but apart from that calling Java code does not differ in any way from calling Scala code. The last step of sitemap processing is invoking the result of the reduction. It's SitemapInvoker[5] class that is responsible for that. This class is only partly implemented as I didn't have time to deal with component factories needed to build actions and pipeline for execution. Anyway, this class extracts two distinct sets of nodes that we are interested in: * actions * pipeline components From this point, building a pipeline and executing actions is rather trivial task. -- o0o -- That's the big picture of the implementation. Now I'll give a few words of comments why I find my implementation easier to follow. First of all, sitemap processing is divided into a few distinct stages: 1. Sitemap parsing 2. Sitemap reduction 3. Sitemap invocation 3.1 Actions execution 3.2 Pipeline building 3.3 Pipeline execution 3.4 Catching possible exception 4. If exception catched, reduction of handle errors node 5. Handling exception with reduction result What's more, for most of the time code is based on immutable structures which is the biggest advantage. What I found hard with current implementation hard is InvocationImpl passed around that quite a lot of state. If you combine it with InvocationResult switches (cases) that every node has to return in its invoke() method the result isn't really appealing. This sort of resembles mistakes from C2.x where big, mutable objects were passed around leading to unmaintainable code. I'm interested in hearing your opinions on that subject. I know that's rather hard to judge rather big piece of code in unfamiliar language but I'm sure you can easily get some overall feeling about my implementation and ideas behind it. PS. You can download the snapshot from GitHub and compile it using Maven as Scala distributes Maven compiler plug-in. This proves that integration with existing Java code-base can be achieved very easily. PS2. This code is just a first version. I see quite a lot of places where it could be made more concise and more readable but I don't have a time right now to polish it more. Thanks for listening. [1]
