Re: playing with sling pipes

2016-01-15 Thread Nicolas Peltier 
sorry, realised you are actually hitting one component, with 2 subproperties, 
you could achieve this with a write pipe like in [0], assuming the null feature 
of the write pipe works which was not the case until i fixed [1], [2] and [3] 
:-)

will send you below sample & built snapshot in private

Nicolas

[0] {
  "jcr:primaryType": "sling:OrderedFolder",
  "testComp": {
"jcr:primaryType": "sling:Folder",
"expr": "[sling:resourceType=my/test]",
"path": "/content/blah",
"sling:resourceType": "slingPipes/slingQuery"
},
  "write": {
"jcr:primaryType": "sling:OrderedFolder",
"sling:resourceType": "slingPipes/write",
"conf": {
  "jcr:primaryType": "nt:unstructured",
  "test2_prop1": "${null}",
  "test2_prop2": "${null}",
  "test1_prop1": "${null}",
  "test1_prop2": "${null}",
  "test1": {
"jcr:primaryType": "sling:Folder",
"prop1": "${testComp.test1_prop1?testComp.test1_prop1:null}",
"prop2": "${testComp.test1_prop2?testComp.test1_prop2:null}"
},
  "test2": {
"jcr:primaryType": "sling:Folder",
"prop1": "${testComp.test2_prop1?testComp.test2_prop1:null}",
"prop2": "${testComp.test2_prop2?testComp.test2_prop2:null}"
}
  }
}
  }

[1] https://issues.apache.org/jira/browse/SLING-5431
[2] https://issues.apache.org/jira/browse/SLING-5433
[3] https://issues.apache.org/jira/browse/SLING-5434
> On 14 Jan 2016, at 23:05, Nicolas Peltier  wrote:
> 
> Right,
> 
> you need to do 2 different pipes for prop1 and prop2 here, as there are 
> clearly 2 different streams here, and shouldn’t be queued.
> 
> the sequence is in your script calling 2 curl :-), or if you really want to 
> do it in one curl,  you can still create a servlet chaining those 2 pipes:
> 
> Pipe prop1Transform = plumberService.getPipe(resourceProp1Transform);
> Pipe prop2Transform = plumberService.getPipe(resourceProp1Transform);
> prop1Transform.getOutput();
> prop2Transform.getOutput();
> resolver.commit();
> 
> But as you did it here, prop2 pipe is expecting the output of moveProp1.
> 
> another thing: does subNode-1 always exist? If not you might want to create 
> that structure by inserting before the move pipe a PathPipe (with autoSave to 
> true that will slows the process but no real choice here).
> 
> Nicolas
> 
>> On 14 Jan 2016, at 19:48, DIRAN SAMARJIAN  wrote:
>> 
>> Hi Nicolas,
>> 
>> Yes, the issue we are having is not to cut the execution stream if a single 
>> pipe fails. We basically have a grouping of pipes in a container pipe but I 
>> can see by design the container pipe stops execution if any intermediate 
>> pipe fails. Is there an alternative way to group pipes together to be able 
>> to run the entire set regardless if any intermediate steps fail?
>> 
>> Here is the relevant config:
>> 
>> {
>> "jcr:primaryType": "sling:OrderedFolder",
>> "jcr:createdBy": "admin",
>> "jcr:created": "Mon Dec 21 2015 21:04:54 GMT-0800",
>> "name": "Prop transform",
>> "jcr:description": "move some props",
>> "sling:resourceType": "slingPipes/container",
>> "conf": {
>>   "jcr:primaryType": "nt:unstructured",
>>   "prop1": {
>> "jcr:primaryType": "nt:unstructured",
>> "expr": "/jcr:root/content//*[jcr:contains(., 
>> 'base/components/content/testcomp') and @item1_title]",
>> "path": "/content",
>> "sling:resourceType": "slingPipes/xpath"
>> },
>>   "moveProp1": {
>> "jcr:primaryType": "nt:unstructured",
>> "expr": "${path.prop1}/subNode-1/newTitle",
>> "path": "${path.prop1}/item1_title",
>> "sling:resourceType": "slingPipes/mv"
>> },
>>   "prop2": {
>> "jcr:primaryType": "nt:unstructured",
>> "expr": "/jcr:root/content//*[jcr:contains(., 
>> 'base/components/content/testcomp') and @item1_description]",
>> "path": "/content",
>> "sling:resourceType": "slingPipes/xpath"
>> },
>>   "moveProp2": {
>> "jcr:primaryType": "nt:unstructured",
>> "expr": "${path.prop2}/subNode-1/newDescription",
>> "path": "${path.prop2}/item1_description",
>> "sling:resourceType": "slingPipes/mv"
>> }
>>   }
>> }
>> 
>>> On Jan 14, 2016, at 2:00 AM, Nicolas Peltier  wrote:
>>> 
>>> Hey Ruben!
>>> 
>>> including sling users DL here.
>>> 
>>> if i understand it correctly, you wish to move several properties from a 
>>> same node in different things,
>>> 
 Eg if the property does not exist, skip the move?
>>> 
>>> Is the problem you have here not to cut the execution stream if one of the 
>>> property does not exist?
>>> 
>>> Can you share a bit more what you are trying to achieve? Ideally paste your 
>>> pipe in this thread (with .tidy.infinity.json)?
>>> 
>>> Thx,
>>> Nicolas
>>> 
>>> 
 On 14 Jan 2016, at 01:29, Ruben Ruesser  wrote:
 
 Nicolas, 
 
 We’re playing a bit with sling pipes – we have a couple of move operations 
 for properties but see that sling pipes

Re: playing with sling pipes

2016-01-15 Thread DIRAN SAMARJIAN 
Thanks Nicolas, the sample config worked great. The patches from the bug fixes 
aren’t in the latest version on github so I had to apply them myself. However, 
I had to remove the patch from [3] because it introduced a bug where only the 
null properties from the first node found are removed. Other than that it 
performed very well, I used a dataset of 50,000 nodes and it finished within 5 
minutes.

On Jan 15, 2016, at 8:18 AM, Nicolas Peltier  wrote:

sorry, realised you are actually hitting one component, with 2 subproperties, 
you could achieve this with a write pipe like in [0], assuming the null feature 
of the write pipe works which was not the case until i fixed [1], [2] and [3] 
:-)

will send you below sample & built snapshot in private

Nicolas

[0] {
"jcr:primaryType": "sling:OrderedFolder",
"testComp": {
  "jcr:primaryType": "sling:Folder",
  "expr": "[sling:resourceType=my/test]",
  "path": "/content/blah",
  "sling:resourceType": "slingPipes/slingQuery"
  },
"write": {
  "jcr:primaryType": "sling:OrderedFolder",
  "sling:resourceType": "slingPipes/write",
  "conf": {
"jcr:primaryType": "nt:unstructured",
"test2_prop1": "${null}",
"test2_prop2": "${null}",
"test1_prop1": "${null}",
"test1_prop2": "${null}",
"test1": {
  "jcr:primaryType": "sling:Folder",
  "prop1": "${testComp.test1_prop1?testComp.test1_prop1:null}",
  "prop2": "${testComp.test1_prop2?testComp.test1_prop2:null}"
  },
"test2": {
  "jcr:primaryType": "sling:Folder",
  "prop1": "${testComp.test2_prop1?testComp.test2_prop1:null}",
  "prop2": "${testComp.test2_prop2?testComp.test2_prop2:null}"
  }
}
  }
}

[1] https://issues.apache.org/jira/browse/SLING-5431
[2] https://issues.apache.org/jira/browse/SLING-5433
[3] https://issues.apache.org/jira/browse/SLING-5434
On 14 Jan 2016, at 23:05, Nicolas Peltier  wrote:

Right,

you need to do 2 different pipes for prop1 and prop2 here, as there are clearly 
2 different streams here, and shouldn’t be queued.

the sequence is in your script calling 2 curl :-), or if you really want to do 
it in one curl,  you can still create a servlet chaining those 2 pipes:

Pipe prop1Transform = plumberService.getPipe(resourceProp1Transform);
Pipe prop2Transform = plumberService.getPipe(resourceProp1Transform);
prop1Transform.getOutput();
prop2Transform.getOutput();
resolver.commit();

But as you did it here, prop2 pipe is expecting the output of moveProp1.

another thing: does subNode-1 always exist? If not you might want to create 
that structure by inserting before the move pipe a PathPipe (with autoSave to 
true that will slows the process but no real choice here).

Nicolas

On 14 Jan 2016, at 19:48, DIRAN SAMARJIAN  wrote:

Hi Nicolas,

Yes, the issue we are having is not to cut the execution stream if a single 
pipe fails. We basically have a grouping of pipes in a container pipe but I can 
see by design the container pipe stops execution if any intermediate pipe 
fails. Is there an alternative way to group pipes together to be able to run 
the entire set regardless if any intermediate steps fail?

Here is the relevant config:

{
"jcr:primaryType": "sling:OrderedFolder",
"jcr:createdBy": "admin",
"jcr:created": "Mon Dec 21 2015 21:04:54 GMT-0800",
"name": "Prop transform",
"jcr:description": "move some props",
"sling:resourceType": "slingPipes/container",
"conf": {
"jcr:primaryType": "nt:unstructured",
"prop1": {
  "jcr:primaryType": "nt:unstructured",
  "expr": "/jcr:root/content//*[jcr:contains(., 
'base/components/content/testcomp') and @item1_title]",
  "path": "/content",
  "sling:resourceType": "slingPipes/xpath"
  },
"moveProp1": {
  "jcr:primaryType": "nt:unstructured",
  "expr": "${path.prop1}/subNode-1/newTitle",
  "path": "${path.prop1}/item1_title",
  "sling:resourceType": "slingPipes/mv"
  },
"prop2": {
  "jcr:primaryType": "nt:unstructured",
  "expr": "/jcr:root/content//*[jcr:contains(., 
'base/components/content/testcomp') and @item1_description]",
  "path": "/content",
  "sling:resourceType": "slingPipes/xpath"
  },
"moveProp2": {
  "jcr:primaryType": "nt:unstructured",
  "expr": "${path.prop2}/subNode-1/newDescription",
  "path": "${path.prop2}/item1_description",
  "sling:resourceType": "slingPipes/mv"
  }
}
}

On Jan 14, 2016, at 2:00 AM, Nicolas Peltier  wrote:

Hey Ruben!

including sling users DL here.

if i understand it correctly, you wish to move several properties from a same 
node in different things,

Eg if the property does not exist, skip the move?

Is the problem you have here not to cut the execution stream if one of the 
property does not exist?

Can you share a bit more what you are trying to achieve? Ideally paste your 
pipe in this thread (with .tidy.infinity.json)?

Thx,
Nicolas


On 14 Jan 2016, at 01:29, Ruben Ruesser  wrote:

Nicolas,

We’re playing a bit with sling pipes – we have a couple of

Re: playing with sling pipes

2016-01-15 Thread Nicolas Peltier 
Thx Diran! Updated [3]'s patch with your feedback

On 15 janv. 2016, at 23:54, DIRAN SAMARJIAN 
> wrote:

Thanks Nicolas, the sample config worked great. The patches from the bug fixes 
aren’t in the latest version on github so I had to apply them myself. However, 
I had to remove the patch from [3] because it introduced a bug where only the 
null properties from the first node found are removed. Other than that it 
performed very well, I used a dataset of 50,000 nodes and it finished within 5 
minutes.

On Jan 15, 2016, at 8:18 AM, Nicolas Peltier 
> wrote:

sorry, realised you are actually hitting one component, with 2 subproperties, 
you could achieve this with a write pipe like in [0], assuming the null feature 
of the write pipe works which was not the case until i fixed [1], [2] and [3] 
:-)

will send you below sample & built snapshot in private

Nicolas

[0] {
"jcr:primaryType": "sling:OrderedFolder",
"testComp": {
  "jcr:primaryType": "sling:Folder",
  "expr": "[sling:resourceType=my/test]",
  "path": "/content/blah",
  "sling:resourceType": "slingPipes/slingQuery"
  },
"write": {
  "jcr:primaryType": "sling:OrderedFolder",
  "sling:resourceType": "slingPipes/write",
  "conf": {
"jcr:primaryType": "nt:unstructured",
"test2_prop1": "${null}",
"test2_prop2": "${null}",
"test1_prop1": "${null}",
"test1_prop2": "${null}",
"test1": {
  "jcr:primaryType": "sling:Folder",
  "prop1": "${testComp.test1_prop1?testComp.test1_prop1:null}",
  "prop2": "${testComp.test1_prop2?testComp.test1_prop2:null}"
  },
"test2": {
  "jcr:primaryType": "sling:Folder",
  "prop1": "${testComp.test2_prop1?testComp.test2_prop1:null}",
  "prop2": "${testComp.test2_prop2?testComp.test2_prop2:null}"
  }
}
  }
}

[1] https://issues.apache.org/jira/browse/SLING-5431
[2] https://issues.apache.org/jira/browse/SLING-5433
[3] https://issues.apache.org/jira/browse/SLING-5434
On 14 Jan 2016, at 23:05, Nicolas Peltier 
> wrote:

Right,

you need to do 2 different pipes for prop1 and prop2 here, as there are clearly 
2 different streams here, and shouldn’t be queued.

the sequence is in your script calling 2 curl :-), or if you really want to do 
it in one curl,  you can still create a servlet chaining those 2 pipes:

Pipe prop1Transform = plumberService.getPipe(resourceProp1Transform);
Pipe prop2Transform = plumberService.getPipe(resourceProp1Transform);
prop1Transform.getOutput();
prop2Transform.getOutput();
resolver.commit();

But as you did it here, prop2 pipe is expecting the output of moveProp1.

another thing: does subNode-1 always exist? If not you might want to create 
that structure by inserting before the move pipe a PathPipe (with autoSave to 
true that will slows the process but no real choice here).

Nicolas

On 14 Jan 2016, at 19:48, DIRAN SAMARJIAN 
> wrote:

Hi Nicolas,

Yes, the issue we are having is not to cut the execution stream if a single 
pipe fails. We basically have a grouping of pipes in a container pipe but I can 
see by design the container pipe stops execution if any intermediate pipe 
fails. Is there an alternative way to group pipes together to be able to run 
the entire set regardless if any intermediate steps fail?

Here is the relevant config:

{
"jcr:primaryType": "sling:OrderedFolder",
"jcr:createdBy": "admin",
"jcr:created": "Mon Dec 21 2015 21:04:54 GMT-0800",
"name": "Prop transform",
"jcr:description": "move some props",
"sling:resourceType": "slingPipes/container",
"conf": {
"jcr:primaryType": "nt:unstructured",
"prop1": {
  "jcr:primaryType": "nt:unstructured",
  "expr": "/jcr:root/content//*[jcr:contains(., 
'base/components/content/testcomp') and @item1_title]",
  "path": "/content",
  "sling:resourceType": "slingPipes/xpath"
  },
"moveProp1": {
  "jcr:primaryType": "nt:unstructured",
  "expr": "${path.prop1}/subNode-1/newTitle",
  "path": "${path.prop1}/item1_title",
  "sling:resourceType": "slingPipes/mv"
  },
"prop2": {
  "jcr:primaryType": "nt:unstructured",
  "expr": "/jcr:root/content//*[jcr:contains(., 
'base/components/content/testcomp') and @item1_description]",
  "path": "/content",
  "sling:resourceType": "slingPipes/xpath"
  },
"moveProp2": {
  "jcr:primaryType": "nt:unstructured",
  "expr": "${path.prop2}/subNode-1/newDescription",
  "path": "${path.prop2}/item1_description",
  "sling:resourceType": "slingPipes/mv"
  }
}
}

On Jan 14, 2016, at 2:00 AM, Nicolas Peltier 
> wrote:

Hey Ruben!

including sling users DL here.

if i understand it correctly, you wish to move several properties from a same 
node in different things,

Eg if the property does not exist, skip the move?

Is the problem you have here not to cut the execution stream if one of the 
property does not exist?

Can you share a

Re: playing with sling pipes

2016-01-14 Thread Nicolas Peltier 
Hey Ruben!

including sling users DL here.

if i understand it correctly, you wish to move several properties from a same 
node in different things,

>  Eg if the property does not exist, skip the move?

Is the problem you have here not to cut the execution stream if one of the 
property does not exist?

Can you share a bit more what you are trying to achieve? Ideally paste your 
pipe in this thread (with .tidy.infinity.json)?

Thx,
Nicolas


> On 14 Jan 2016, at 01:29, Ruben Ruesser  wrote:
> 
> Nicolas, 
>  
> We’re playing a bit with sling pipes – we have a couple of move operations 
> for properties but see that sling pipes stops whenever a property does not 
> exist. Is there a way to do a conditional move. Eg if the property does not 
> exist, skip the move? 
>  
> thanks
>  
> Ruben Reusser.

Re: playing with sling pipes

2016-01-14 Thread DIRAN SAMARJIAN 
Hi Nicolas,

Yes, the issue we are having is not to cut the execution stream if a single 
pipe fails. We basically have a grouping of pipes in a container pipe but I can 
see by design the container pipe stops execution if any intermediate pipe 
fails. Is there an alternative way to group pipes together to be able to run 
the entire set regardless if any intermediate steps fail?

Here is the relevant config:

{
 "jcr:primaryType": "sling:OrderedFolder",
 "jcr:createdBy": "admin",
 "jcr:created": "Mon Dec 21 2015 21:04:54 GMT-0800",
 "name": "Prop transform",
 "jcr:description": "move some props",
 "sling:resourceType": "slingPipes/container",
 "conf": {
   "jcr:primaryType": "nt:unstructured",
   "prop1": {
 "jcr:primaryType": "nt:unstructured",
 "expr": "/jcr:root/content//*[jcr:contains(., 
'base/components/content/testcomp') and @item1_title]",
 "path": "/content",
 "sling:resourceType": "slingPipes/xpath"
 },
   "moveProp1": {
 "jcr:primaryType": "nt:unstructured",
 "expr": "${path.prop1}/subNode-1/newTitle",
 "path": "${path.prop1}/item1_title",
 "sling:resourceType": "slingPipes/mv"
 },
   "prop2": {
 "jcr:primaryType": "nt:unstructured",
 "expr": "/jcr:root/content//*[jcr:contains(., 
'base/components/content/testcomp') and @item1_description]",
 "path": "/content",
 "sling:resourceType": "slingPipes/xpath"
 },
   "moveProp2": {
 "jcr:primaryType": "nt:unstructured",
 "expr": "${path.prop2}/subNode-1/newDescription",
 "path": "${path.prop2}/item1_description",
 "sling:resourceType": "slingPipes/mv"
 }
   }
 }

On Jan 14, 2016, at 2:00 AM, Nicolas Peltier  wrote:

Hey Ruben!

including sling users DL here.

if i understand it correctly, you wish to move several properties from a same 
node in different things,

Eg if the property does not exist, skip the move?

Is the problem you have here not to cut the execution stream if one of the 
property does not exist?

Can you share a bit more what you are trying to achieve? Ideally paste your 
pipe in this thread (with .tidy.infinity.json)?

Thx,
Nicolas


On 14 Jan 2016, at 01:29, Ruben Ruesser  wrote:

Nicolas,

We're playing a bit with sling pipes - we have a couple of move operations for 
properties but see that sling pipes stops whenever a property does not exist. 
Is there a way to do a conditional move. Eg if the property does not exist, 
skip the move?

thanks

Ruben Reusser.

Re: playing with sling pipes

2016-01-14 Thread Nicolas Peltier 
Right,

you need to do 2 different pipes for prop1 and prop2 here, as there are clearly 
2 different streams here, and shouldn’t be queued.

the sequence is in your script calling 2 curl :-), or if you really want to do 
it in one curl,  you can still create a servlet chaining those 2 pipes:

Pipe prop1Transform = plumberService.getPipe(resourceProp1Transform);
Pipe prop2Transform = plumberService.getPipe(resourceProp1Transform);
prop1Transform.getOutput();
prop2Transform.getOutput();
resolver.commit();

But as you did it here, prop2 pipe is expecting the output of moveProp1.

another thing: does subNode-1 always exist? If not you might want to create 
that structure by inserting before the move pipe a PathPipe (with autoSave to 
true that will slows the process but no real choice here).

Nicolas

> On 14 Jan 2016, at 19:48, DIRAN SAMARJIAN  wrote:
> 
> Hi Nicolas,
> 
> Yes, the issue we are having is not to cut the execution stream if a single 
> pipe fails. We basically have a grouping of pipes in a container pipe but I 
> can see by design the container pipe stops execution if any intermediate pipe 
> fails. Is there an alternative way to group pipes together to be able to run 
> the entire set regardless if any intermediate steps fail?
> 
> Here is the relevant config:
> 
> {
>  "jcr:primaryType": "sling:OrderedFolder",
>  "jcr:createdBy": "admin",
>  "jcr:created": "Mon Dec 21 2015 21:04:54 GMT-0800",
>  "name": "Prop transform",
>  "jcr:description": "move some props",
>  "sling:resourceType": "slingPipes/container",
>  "conf": {
>"jcr:primaryType": "nt:unstructured",
>"prop1": {
>  "jcr:primaryType": "nt:unstructured",
>  "expr": "/jcr:root/content//*[jcr:contains(., 
> 'base/components/content/testcomp') and @item1_title]",
>  "path": "/content",
>  "sling:resourceType": "slingPipes/xpath"
>  },
>"moveProp1": {
>  "jcr:primaryType": "nt:unstructured",
>  "expr": "${path.prop1}/subNode-1/newTitle",
>  "path": "${path.prop1}/item1_title",
>  "sling:resourceType": "slingPipes/mv"
>  },
>"prop2": {
>  "jcr:primaryType": "nt:unstructured",
>  "expr": "/jcr:root/content//*[jcr:contains(., 
> 'base/components/content/testcomp') and @item1_description]",
>  "path": "/content",
>  "sling:resourceType": "slingPipes/xpath"
>  },
>"moveProp2": {
>  "jcr:primaryType": "nt:unstructured",
>  "expr": "${path.prop2}/subNode-1/newDescription",
>  "path": "${path.prop2}/item1_description",
>  "sling:resourceType": "slingPipes/mv"
>  }
>}
>  }
> 
>> On Jan 14, 2016, at 2:00 AM, Nicolas Peltier  wrote:
>> 
>> Hey Ruben!
>> 
>> including sling users DL here.
>> 
>> if i understand it correctly, you wish to move several properties from a 
>> same node in different things,
>> 
>>> Eg if the property does not exist, skip the move?
>> 
>> Is the problem you have here not to cut the execution stream if one of the 
>> property does not exist?
>> 
>> Can you share a bit more what you are trying to achieve? Ideally paste your 
>> pipe in this thread (with .tidy.infinity.json)?
>> 
>> Thx,
>> Nicolas
>> 
>> 
>>> On 14 Jan 2016, at 01:29, Ruben Ruesser  wrote:
>>> 
>>> Nicolas, 
>>> 
>>> We’re playing a bit with sling pipes – we have a couple of move operations 
>>> for properties but see that sling pipes stops whenever a property does not 
>>> exist. Is there a way to do a conditional move. Eg if the property does not 
>>> exist, skip the move? 
>>> 
>>> thanks
>>> 
>>> Ruben Reusser.