[jira] [Commented] (OOZIE-2829) Improve sharelib upload to accept multiple source folders
[
https://issues.apache.org/jira/browse/OOZIE-2829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16561866#comment-16561866
]
Peter Cseh commented on OOZIE-2829:
---
+1
> Improve sharelib upload to accept multiple source folders
> -
>
> Key: OOZIE-2829
> URL: https://issues.apache.org/jira/browse/OOZIE-2829
> Project: Oozie
> Issue Type: Bug
> Components: tools
>Affects Versions: 5.0.0
>Reporter: Peter Cseh
>Assignee: Julia Kinga Marton
>Priority: Major
> Fix For: 5.1.0
>
> Attachments: OOZIE-2829-006.patch, OOZIE-2829-007.patch,
> OOZIE-2829-008.patch, OOZIE-2829-009.patch, OOZIE-2829-01.patch,
> OOZIE-2829-010.patch, OOZIE-2829-011.patch, OOZIE-2829-02.patch,
> OOZIE-2829-03.patch, OOZIE-2829-04.patch, OOZIE-2829-05.patch
>
>
> Right now sharelib can be created via {{sharelib create -fs FS_URI -locallib
> SHARED_LIBRARY}} where the SHARED_LIBRARY can be a tarbal or a folder.
> It would be nice to have the possibility to define additional folders to be
> uploaded into the sharelib, so the users don't have to copy or link the files
> together on their machine.
> The syntax could be something like -additional-lib
> sharelibName=/path/to/source/;/path/to/some/file,sharelibName2=/path/to/some/folder
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-2829) Improve sharelib upload to accept multiple source folders
[
https://issues.apache.org/jira/browse/OOZIE-2829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16558254#comment-16558254
]
Hadoop QA commented on OOZIE-2829:
--
Testing JIRA OOZIE-2829
Cleaning local git workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than
132
.{color:green}+1{color} the patch adds/modifies 6 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warning(s)
.{color:red}WARNING{color}: the current HEAD has 100 Javadoc warning(s)
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
error(s)
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:orange}0{color} There are [1] new bugs found in total that would be nice
to have fixed.
. {color:green}+1{color} There are no new bugs found in [examples].
. {color:green}+1{color} There are no new bugs found in [webapp].
. {color:green}+1{color} There are no new bugs found in [core].
. {color:orange}0{color} There are [1] new bugs found in [tools] that would be
nice to have fixed.
. You can find the FindBugs diff here: tools/findbugs-new.html
. {color:green}+1{color} There are no new bugs found in
[fluent-job/fluent-job-api].
. {color:green}+1{color} There are no new bugs found in [server].
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:green}+1{color} There are no new bugs found in [sharelib/oozie].
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
. {color:green}+1{color} There are no new bugs found in [client].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
.Tests run: 2911
.{color:orange}Tests failed at first run:{color}
TestSLAEmailEventListener#setUp
TestSLAEmailEventListener#setUp
.For the complete list of flaky tests, see TEST-SUMMARY-FULL files.
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch
{color:green}*+1 Overall result, good!, no -1s*{color}
{color:red}. There is at least one warning, please check{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/PreCommit-OOZIE-Build/690/
> Improve sharelib upload to accept multiple source folders
> -
>
> Key: OOZIE-2829
> URL: https://issues.apache.org/jira/browse/OOZIE-2829
> Project: Oozie
> Issue Type: Bug
> Components: tools
>Affects Versions: 5.0.0
>Reporter: Peter Cseh
>Assignee: Julia Kinga Marton
>Priority: Major
> Fix For: 5.1.0
>
> Attachments: OOZIE-2829-006.patch, OOZIE-2829-007.patch,
> OOZIE-2829-008.patch, OOZIE-2829-009.patch, OOZIE-2829-01.patch,
> OOZIE-2829-010.patch, OOZIE-2829-011.patch, OOZIE-2829-02.patch,
> OOZIE-2829-03.patch, OOZIE-2829-04.patch, OOZIE-2829-05.patch
>
>
> Right now sharelib can be created via {{sharelib create -fs FS_URI -locallib
> SHARED_LIBRARY}} where the SHARED_LIBRARY can be a tarbal or a folder.
> It would be nice to have the possibility to define additional folders to be
> uploaded into the sharelib, so the users don't have to copy or link the files
> together on their machine.
> The syntax could be something like -additional-lib
> sharelibName=/path/to/source/;/path/to/some/file,sharelibName2=/path/to/some/folder
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-2829) Improve sharelib upload to accept multiple source folders
[
https://issues.apache.org/jira/browse/OOZIE-2829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16558181#comment-16558181
]
Hadoop QA commented on OOZIE-2829:
--
PreCommit-OOZIE-Build started
> Improve sharelib upload to accept multiple source folders
> -
>
> Key: OOZIE-2829
> URL: https://issues.apache.org/jira/browse/OOZIE-2829
> Project: Oozie
> Issue Type: Bug
> Components: tools
>Affects Versions: 5.0.0
>Reporter: Peter Cseh
>Assignee: Julia Kinga Marton
>Priority: Major
> Fix For: 5.1.0
>
> Attachments: OOZIE-2829-006.patch, OOZIE-2829-007.patch,
> OOZIE-2829-008.patch, OOZIE-2829-009.patch, OOZIE-2829-01.patch,
> OOZIE-2829-010.patch, OOZIE-2829-011.patch, OOZIE-2829-02.patch,
> OOZIE-2829-03.patch, OOZIE-2829-04.patch, OOZIE-2829-05.patch
>
>
> Right now sharelib can be created via {{sharelib create -fs FS_URI -locallib
> SHARED_LIBRARY}} where the SHARED_LIBRARY can be a tarbal or a folder.
> It would be nice to have the possibility to define additional folders to be
> uploaded into the sharelib, so the users don't have to copy or link the files
> together on their machine.
> The syntax could be something like -additional-lib
> sharelibName=/path/to/source/;/path/to/some/file,sharelibName2=/path/to/some/folder
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-2829) Improve sharelib upload to accept multiple source folders
[
https://issues.apache.org/jira/browse/OOZIE-2829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16552818#comment-16552818
]
Peter Cseh commented on OOZIE-2829:
---
How about having multiple options for multiple sharelibs?
{quote}
-extralibs sharelibName=/path/to/source/,/path/to/some/file -extralibs
sharelibName2=/path/to/some/folder,hdfs://my/jar.jar#myjar.jar
{quote}
> Improve sharelib upload to accept multiple source folders
> -
>
> Key: OOZIE-2829
> URL: https://issues.apache.org/jira/browse/OOZIE-2829
> Project: Oozie
> Issue Type: Bug
> Components: tools
>Affects Versions: 5.0.0
>Reporter: Peter Cseh
>Assignee: Julia Kinga Marton
>Priority: Major
> Fix For: 5.1.0
>
> Attachments: OOZIE-2829-006.patch, OOZIE-2829-007.patch,
> OOZIE-2829-008.patch, OOZIE-2829-009.patch, OOZIE-2829-01.patch,
> OOZIE-2829-010.patch, OOZIE-2829-02.patch, OOZIE-2829-03.patch,
> OOZIE-2829-04.patch, OOZIE-2829-05.patch
>
>
> Right now sharelib can be created via {{sharelib create -fs FS_URI -locallib
> SHARED_LIBRARY}} where the SHARED_LIBRARY can be a tarbal or a folder.
> It would be nice to have the possibility to define additional folders to be
> uploaded into the sharelib, so the users don't have to copy or link the files
> together on their machine.
> The syntax could be something like -additional-lib
> sharelibName=/path/to/source/;/path/to/some/file,sharelibName2=/path/to/some/folder
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-2829) Improve sharelib upload to accept multiple source folders
[
https://issues.apache.org/jira/browse/OOZIE-2829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16552048#comment-16552048
]
Hadoop QA commented on OOZIE-2829:
--
Testing JIRA OOZIE-2829
Cleaning local git workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than
132
.{color:green}+1{color} the patch adds/modifies 6 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warning(s)
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
error(s)
.{color:red}ERROR{color}: the current HEAD has 2 Javadoc error(s)
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:orange}0{color} There are [1] new bugs found in total that would be nice
to have fixed.
. {color:green}+1{color} There are no new bugs found in [server].
. {color:green}+1{color} There are no new bugs found in [client].
. {color:green}+1{color} There are no new bugs found in [core].
. {color:green}+1{color} There are no new bugs found in
[fluent-job/fluent-job-api].
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:orange}0{color} There are [1] new bugs found in [tools] that would be
nice to have fixed.
. You can find the FindBugs diff here: tools/findbugs-new.html
. {color:green}+1{color} There are no new bugs found in [examples].
. {color:green}+1{color} There are no new bugs found in [webapp].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [sharelib/oozie].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:red}-1 TESTS{color}
.Tests run: 2912
.Tests failed: 1
.Tests errors: 0
.The patch failed the following testcases:
testCopyingExtraSharelibs(org.apache.oozie.tools.TestOozieSharelibCLI)
.Tests failing with errors:
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch
{color:red}*-1 Overall result, please check the reported -1(s)*{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/PreCommit-OOZIE-Build/680/
> Improve sharelib upload to accept multiple source folders
> -
>
> Key: OOZIE-2829
> URL: https://issues.apache.org/jira/browse/OOZIE-2829
> Project: Oozie
> Issue Type: Bug
> Components: tools
>Affects Versions: 5.0.0
>Reporter: Peter Cseh
>Assignee: Julia Kinga Marton
>Priority: Major
> Fix For: 5.1.0
>
> Attachments: OOZIE-2829-006.patch, OOZIE-2829-007.patch,
> OOZIE-2829-008.patch, OOZIE-2829-009.patch, OOZIE-2829-01.patch,
> OOZIE-2829-02.patch, OOZIE-2829-03.patch, OOZIE-2829-04.patch,
> OOZIE-2829-05.patch
>
>
> Right now sharelib can be created via {{sharelib create -fs FS_URI -locallib
> SHARED_LIBRARY}} where the SHARED_LIBRARY can be a tarbal or a folder.
> It would be nice to have the possibility to define additional folders to be
> uploaded into the sharelib, so the users don't have to copy or link the files
> together on their machine.
> The syntax could be something like -additional-lib
> sharelibName=/path/to/source/;/path/to/some/file,sharelibName2=/path/to/some/folder
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-2829) Improve sharelib upload to accept multiple source folders
[
https://issues.apache.org/jira/browse/OOZIE-2829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16552015#comment-16552015
]
Hadoop QA commented on OOZIE-2829:
--
PreCommit-OOZIE-Build started
> Improve sharelib upload to accept multiple source folders
> -
>
> Key: OOZIE-2829
> URL: https://issues.apache.org/jira/browse/OOZIE-2829
> Project: Oozie
> Issue Type: Bug
> Components: tools
>Affects Versions: 5.0.0
>Reporter: Peter Cseh
>Assignee: Julia Kinga Marton
>Priority: Major
> Fix For: 5.1.0
>
> Attachments: OOZIE-2829-006.patch, OOZIE-2829-007.patch,
> OOZIE-2829-008.patch, OOZIE-2829-009.patch, OOZIE-2829-01.patch,
> OOZIE-2829-02.patch, OOZIE-2829-03.patch, OOZIE-2829-04.patch,
> OOZIE-2829-05.patch
>
>
> Right now sharelib can be created via {{sharelib create -fs FS_URI -locallib
> SHARED_LIBRARY}} where the SHARED_LIBRARY can be a tarbal or a folder.
> It would be nice to have the possibility to define additional folders to be
> uploaded into the sharelib, so the users don't have to copy or link the files
> together on their machine.
> The syntax could be something like -additional-lib
> sharelibName=/path/to/source/;/path/to/some/file,sharelibName2=/path/to/some/folder
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-2829) Improve sharelib upload to accept multiple source folders
[
https://issues.apache.org/jira/browse/OOZIE-2829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16549393#comment-16549393
]
Julia Kinga Marton commented on OOZIE-2829:
---
What about using '+' as sharelibs delimiter? The problem is that every
character that seems to be logic as a delimiter, is interpreted by the Shell.
Using '+' would result in the following command:
{noformat}
-additional-lib
sharelibName=/path/to/source/,/path/to/some/file+sharelibName2=/path/to/some/folder,hdfs://my/jar.jar#myjar.jar{noformat}
> Improve sharelib upload to accept multiple source folders
> -
>
> Key: OOZIE-2829
> URL: https://issues.apache.org/jira/browse/OOZIE-2829
> Project: Oozie
> Issue Type: Bug
> Components: tools
>Affects Versions: 5.0.0
>Reporter: Peter Cseh
>Assignee: Julia Kinga Marton
>Priority: Major
> Fix For: 5.1.0
>
> Attachments: OOZIE-2829-006.patch, OOZIE-2829-007.patch,
> OOZIE-2829-008.patch, OOZIE-2829-01.patch, OOZIE-2829-02.patch,
> OOZIE-2829-03.patch, OOZIE-2829-04.patch, OOZIE-2829-05.patch
>
>
> Right now sharelib can be created via {{sharelib create -fs FS_URI -locallib
> SHARED_LIBRARY}} where the SHARED_LIBRARY can be a tarbal or a folder.
> It would be nice to have the possibility to define additional folders to be
> uploaded into the sharelib, so the users don't have to copy or link the files
> together on their machine.
> The syntax could be something like -additional-lib
> sharelibName=/path/to/source/;/path/to/some/file,sharelibName2=/path/to/some/folder
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-2829) Improve sharelib upload to accept multiple source folders
[
https://issues.apache.org/jira/browse/OOZIE-2829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16549323#comment-16549323
]
Julia Kinga Marton commented on OOZIE-2829:
---
I agree with [~asasvari], that for one extra sharelib we should use a comma
separated list of files.
I am thinking of what we should use as a separator between the extra sharelibs.
None of the following will work: &, ;, :, |. I will try with separating them
with a simple space.
> Improve sharelib upload to accept multiple source folders
> -
>
> Key: OOZIE-2829
> URL: https://issues.apache.org/jira/browse/OOZIE-2829
> Project: Oozie
> Issue Type: Bug
> Components: tools
>Affects Versions: 5.0.0
>Reporter: Peter Cseh
>Assignee: Julia Kinga Marton
>Priority: Major
> Fix For: 5.1.0
>
> Attachments: OOZIE-2829-006.patch, OOZIE-2829-007.patch,
> OOZIE-2829-008.patch, OOZIE-2829-01.patch, OOZIE-2829-02.patch,
> OOZIE-2829-03.patch, OOZIE-2829-04.patch, OOZIE-2829-05.patch
>
>
> Right now sharelib can be created via {{sharelib create -fs FS_URI -locallib
> SHARED_LIBRARY}} where the SHARED_LIBRARY can be a tarbal or a folder.
> It would be nice to have the possibility to define additional folders to be
> uploaded into the sharelib, so the users don't have to copy or link the files
> together on their machine.
> The syntax could be something like -additional-lib
> sharelibName=/path/to/source/;/path/to/some/file,sharelibName2=/path/to/some/folder
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-2829) Improve sharelib upload to accept multiple source folders
[
https://issues.apache.org/jira/browse/OOZIE-2829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16549315#comment-16549315
]
Attila Sasvari commented on OOZIE-2829:
---
If you look at how Hadoop components are handling/pass multiple files/jars, you
pretty often see a comma separated list of files:
- YARN / Map-Reduce: {{--libjars}}:
https://hadoop.apache.org/docs/r2.8.2/hadoop-project-dist/hadoop-common/CommandsManual.html#Generic_Options
- Apache Spark: spark-submit
https://spark.apache.org/docs/latest/submitting-applications.html
- Apache Pig - pig.additional.jars.uris
- Apache Oozie - Sharelib mapping file:
https://oozie.apache.org/docs/5.0.0/AG_Install.html#Oozie_Share_Lib
I think it would be better to specify a comma separated list of files for a
given extra sharelib.
> Improve sharelib upload to accept multiple source folders
> -
>
> Key: OOZIE-2829
> URL: https://issues.apache.org/jira/browse/OOZIE-2829
> Project: Oozie
> Issue Type: Bug
> Components: tools
>Affects Versions: 5.0.0
>Reporter: Peter Cseh
>Assignee: Julia Kinga Marton
>Priority: Major
> Fix For: 5.1.0
>
> Attachments: OOZIE-2829-006.patch, OOZIE-2829-007.patch,
> OOZIE-2829-008.patch, OOZIE-2829-01.patch, OOZIE-2829-02.patch,
> OOZIE-2829-03.patch, OOZIE-2829-04.patch, OOZIE-2829-05.patch
>
>
> Right now sharelib can be created via {{sharelib create -fs FS_URI -locallib
> SHARED_LIBRARY}} where the SHARED_LIBRARY can be a tarbal or a folder.
> It would be nice to have the possibility to define additional folders to be
> uploaded into the sharelib, so the users don't have to copy or link the files
> together on their machine.
> The syntax could be something like -additional-lib
> sharelibName=/path/to/source/;/path/to/some/file,sharelibName2=/path/to/some/folder
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-2829) Improve sharelib upload to accept multiple source folders
[
https://issues.apache.org/jira/browse/OOZIE-2829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16549233#comment-16549233
]
Andras Piros commented on OOZIE-2829:
-
[~kmarton] I'm OK using colon {{,}} as the separator between sharelib name-path
pairs, and pipe {{|}} between sharelib paths like this:
{noformat}
additional-lib
sharelibName1=/path/to/source/|/path/to/some/file,sharelibName2=/path/to/some/folder
{noformat}
> Improve sharelib upload to accept multiple source folders
> -
>
> Key: OOZIE-2829
> URL: https://issues.apache.org/jira/browse/OOZIE-2829
> Project: Oozie
> Issue Type: Bug
> Components: tools
>Affects Versions: 5.0.0
>Reporter: Peter Cseh
>Assignee: Julia Kinga Marton
>Priority: Major
> Fix For: 5.1.0
>
> Attachments: OOZIE-2829-006.patch, OOZIE-2829-007.patch,
> OOZIE-2829-008.patch, OOZIE-2829-01.patch, OOZIE-2829-02.patch,
> OOZIE-2829-03.patch, OOZIE-2829-04.patch, OOZIE-2829-05.patch
>
>
> Right now sharelib can be created via {{sharelib create -fs FS_URI -locallib
> SHARED_LIBRARY}} where the SHARED_LIBRARY can be a tarbal or a folder.
> It would be nice to have the possibility to define additional folders to be
> uploaded into the sharelib, so the users don't have to copy or link the files
> together on their machine.
> The syntax could be something like -additional-lib
> sharelibName=/path/to/source/;/path/to/some/file,sharelibName2=/path/to/some/folder
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-2829) Improve sharelib upload to accept multiple source folders
[
https://issues.apache.org/jira/browse/OOZIE-2829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16549228#comment-16549228
]
Julia Kinga Marton commented on OOZIE-2829:
---
Using colon as sharelib separator is not a good idea, because as
[~andras.piros] pointed out, if we have an hdfs path
([hdfs://my/jar.jar#myjar.jar]) would cause some troubles. Using semicolon is
also a bad choice in Linux. What about using &?
This way the command would be:
-additional-lib
sharelibName=/path/to/source/,/path/to/some/file=/path/to/some/folder
Any other better ideas are welcome :)
> Improve sharelib upload to accept multiple source folders
> -
>
> Key: OOZIE-2829
> URL: https://issues.apache.org/jira/browse/OOZIE-2829
> Project: Oozie
> Issue Type: Bug
> Components: tools
>Affects Versions: 5.0.0
>Reporter: Peter Cseh
>Assignee: Julia Kinga Marton
>Priority: Major
> Fix For: 5.1.0
>
> Attachments: OOZIE-2829-006.patch, OOZIE-2829-007.patch,
> OOZIE-2829-008.patch, OOZIE-2829-01.patch, OOZIE-2829-02.patch,
> OOZIE-2829-03.patch, OOZIE-2829-04.patch, OOZIE-2829-05.patch
>
>
> Right now sharelib can be created via {{sharelib create -fs FS_URI -locallib
> SHARED_LIBRARY}} where the SHARED_LIBRARY can be a tarbal or a folder.
> It would be nice to have the possibility to define additional folders to be
> uploaded into the sharelib, so the users don't have to copy or link the files
> together on their machine.
> The syntax could be something like -additional-lib
> sharelibName=/path/to/source/;/path/to/some/file,sharelibName2=/path/to/some/folder
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-2829) Improve sharelib upload to accept multiple source folders
[
https://issues.apache.org/jira/browse/OOZIE-2829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16534946#comment-16534946
]
Hadoop QA commented on OOZIE-2829:
--
Testing JIRA OOZIE-2829
Cleaning local git workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than
132
.{color:green}+1{color} the patch adds/modifies 5 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warning(s)
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
error(s)
.{color:red}ERROR{color}: the current HEAD has 2 Javadoc error(s)
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:red}-1{color} There are [1] new bugs found below threshold in total that
must be fixed.
. {color:green}+1{color} There are no new bugs found in [examples].
. {color:green}+1{color} There are no new bugs found in [core].
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:green}+1{color} There are no new bugs found in [sharelib/oozie].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [webapp].
. {color:red}-1{color} There are [1] new bugs found below threshold in [tools]
that must be fixed.
. You can find the FindBugs diff here (look for the red and orange ones):
tools/findbugs-new.html
. The most important FindBugs errors are:
. At OozieSharelibCLI.java:[line 182]:
java/io/File.init(Ljava/lang/String;)V reads a file whose location
might be specified by user input
. At OozieSharelibCLI.java:[line 182]: At OozieSharelibCLI.java:[line 174]
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:green}+1{color} There are no new bugs found in [server].
. {color:green}+1{color} There are no new bugs found in
[fluent-job/fluent-job-api].
. {color:green}+1{color} There are no new bugs found in [client].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
.Tests run: 2910
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch
{color:red}*-1 Overall result, please check the reported -1(s)*{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/PreCommit-OOZIE-Build/659/
> Improve sharelib upload to accept multiple source folders
> -
>
> Key: OOZIE-2829
> URL: https://issues.apache.org/jira/browse/OOZIE-2829
> Project: Oozie
> Issue Type: Bug
> Components: tools
>Affects Versions: 5.0.0
>Reporter: Peter Cseh
>Assignee: Julia Kinga Marton
>Priority: Major
> Fix For: 5.1.0
>
> Attachments: OOZIE-2829-006.patch, OOZIE-2829-007.patch,
> OOZIE-2829-008.patch, OOZIE-2829-01.patch, OOZIE-2829-02.patch,
> OOZIE-2829-03.patch, OOZIE-2829-04.patch, OOZIE-2829-05.patch
>
>
> Right now sharelib can be created via {{sharelib create -fs FS_URI -locallib
> SHARED_LIBRARY}} where the SHARED_LIBRARY can be a tarbal or a folder.
> It would be nice to have the possibility to define additional folders to be
> uploaded into the sharelib, so the users don't have to copy or link the files
> together on their machine.
> The syntax could be something like -additional-lib
> sharelibName=/path/to/source/;/path/to/some/file,sharelibName2=/path/to/some/folder
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-2829) Improve sharelib upload to accept multiple source folders
[
https://issues.apache.org/jira/browse/OOZIE-2829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16534816#comment-16534816
]
Hadoop QA commented on OOZIE-2829:
--
PreCommit-OOZIE-Build started
> Improve sharelib upload to accept multiple source folders
> -
>
> Key: OOZIE-2829
> URL: https://issues.apache.org/jira/browse/OOZIE-2829
> Project: Oozie
> Issue Type: Bug
> Components: tools
>Affects Versions: 5.0.0
>Reporter: Peter Cseh
>Assignee: Julia Kinga Marton
>Priority: Major
> Fix For: 5.1.0
>
> Attachments: OOZIE-2829-006.patch, OOZIE-2829-007.patch,
> OOZIE-2829-008.patch, OOZIE-2829-01.patch, OOZIE-2829-02.patch,
> OOZIE-2829-03.patch, OOZIE-2829-04.patch, OOZIE-2829-05.patch
>
>
> Right now sharelib can be created via {{sharelib create -fs FS_URI -locallib
> SHARED_LIBRARY}} where the SHARED_LIBRARY can be a tarbal or a folder.
> It would be nice to have the possibility to define additional folders to be
> uploaded into the sharelib, so the users don't have to copy or link the files
> together on their machine.
> The syntax could be something like -additional-lib
> sharelibName=/path/to/source/;/path/to/some/file,sharelibName2=/path/to/some/folder
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-2829) Improve sharelib upload to accept multiple source folders
[
https://issues.apache.org/jira/browse/OOZIE-2829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16534665#comment-16534665
]
Hadoop QA commented on OOZIE-2829:
--
Testing JIRA OOZIE-2829
Cleaning local git workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:red}-1{color} the patch contains 4 line(s) longer than 132
characters
.{color:green}+1{color} the patch adds/modifies 5 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warning(s)
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
error(s)
.{color:red}ERROR{color}: the current HEAD has 2 Javadoc error(s)
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:red}-1{color} There are [2] new bugs found below threshold in total that
must be fixed.
. {color:green}+1{color} There are no new bugs found in [examples].
. {color:green}+1{color} There are no new bugs found in [core].
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:green}+1{color} There are no new bugs found in [sharelib/oozie].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [webapp].
. {color:red}-1{color} There are [2] new bugs found below threshold in [tools]
that must be fixed.
. You can find the FindBugs diff here (look for the red and orange ones):
tools/findbugs-new.html
. The most important FindBugs errors are:
. At OozieSharelibCLI.java:[line 273]:
java/io/File.init(Ljava/lang/String;)V reads a file whose location
might be specified by user input
. At OozieSharelibCLI.java:[line 169]: At OozieSharelibCLI.java:[line 168]
. At OozieSharelibCLI.java:[line 270]: At OozieSharelibCLI.java:[line 216]
. java/io/File.init(Ljava/lang/String;)V reads a file whose location
might be specified by user input: At OozieSharelibCLI.java:[line 272]
. At OozieSharelibCLI.java:[line 172]: At OozieSharelibCLI.java:[line 180]
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:green}+1{color} There are no new bugs found in [server].
. {color:green}+1{color} There are no new bugs found in
[fluent-job/fluent-job-api].
. {color:green}+1{color} There are no new bugs found in [client].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:red}-1 TESTS{color}
.Tests run: 2910
.Tests failed: 0
.Tests errors: 2
.The patch failed the following testcases:
.Tests failing with errors:
testFakeCommand(org.apache.oozie.tools.TestOozieSharelibCLI)
testHelpCommand(org.apache.oozie.tools.TestOozieSharelibCLI)
.{color:orange}Tests failed at first run:{color}
TestPurgeXCommand#testPurgeBundleWithCoordChildWithWFChild3
TestActionCheckXCommand#testActionCheck
TestActionStartXCommand#testActionWithEscapedStringAndCDATA
TestJMSJobEventListener#testConnectionDrop
.For the complete list of flaky tests, see TEST-SUMMARY-FULL files.
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch
{color:red}*-1 Overall result, please check the reported -1(s)*{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/PreCommit-OOZIE-Build/658/
> Improve sharelib upload to accept multiple source folders
> -
>
> Key: OOZIE-2829
> URL: https://issues.apache.org/jira/browse/OOZIE-2829
> Project: Oozie
> Issue Type: Bug
> Components: tools
>Affects Versions: 5.0.0
>Reporter: Peter Cseh
>Assignee: Julia Kinga Marton
>
[jira] [Commented] (OOZIE-2829) Improve sharelib upload to accept multiple source folders
[
https://issues.apache.org/jira/browse/OOZIE-2829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16534576#comment-16534576
]
Hadoop QA commented on OOZIE-2829:
--
PreCommit-OOZIE-Build started
> Improve sharelib upload to accept multiple source folders
> -
>
> Key: OOZIE-2829
> URL: https://issues.apache.org/jira/browse/OOZIE-2829
> Project: Oozie
> Issue Type: Bug
> Components: tools
>Affects Versions: 5.0.0
>Reporter: Peter Cseh
>Assignee: Julia Kinga Marton
>Priority: Major
> Fix For: 5.1.0
>
> Attachments: OOZIE-2829-006.patch, OOZIE-2829-007.patch,
> OOZIE-2829-01.patch, OOZIE-2829-02.patch, OOZIE-2829-03.patch,
> OOZIE-2829-04.patch, OOZIE-2829-05.patch
>
>
> Right now sharelib can be created via {{sharelib create -fs FS_URI -locallib
> SHARED_LIBRARY}} where the SHARED_LIBRARY can be a tarbal or a folder.
> It would be nice to have the possibility to define additional folders to be
> uploaded into the sharelib, so the users don't have to copy or link the files
> together on their machine.
> The syntax could be something like -additional-lib
> sharelibName=/path/to/source/;/path/to/some/file,sharelibName2=/path/to/some/folder
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-2829) Improve sharelib upload to accept multiple source folders
[
https://issues.apache.org/jira/browse/OOZIE-2829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16533716#comment-16533716
]
Hadoop QA commented on OOZIE-2829:
--
Testing JIRA OOZIE-2829
Cleaning local git workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:red}-1{color} the patch contains 4 line(s) longer than 132
characters
.{color:green}+1{color} the patch adds/modifies 5 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warning(s)
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
error(s)
.{color:red}ERROR{color}: the current HEAD has 2 Javadoc error(s)
{color:red}-1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:red}-1{color} patch does not compile
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:green}+1{color} There are no new bugs found in total.
. {color:green}+1{color} There are no new bugs found in [examples].
. {color:green}+1{color} There are no new bugs found in [core].
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:green}+1{color} There are no new bugs found in [sharelib/oozie].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [webapp].
. {color:green}+1{color} There are no new bugs found in [tools].
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:green}+1{color} There are no new bugs found in [server].
. {color:green}+1{color} There are no new bugs found in
[fluent-job/fluent-job-api].
. {color:green}+1{color} There are no new bugs found in [client].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:red}-1 TESTS{color} - patch does not compile, cannot run testcases
.{color:orange}Tests failed at first run:{color}
TestPurgeXCommand#testPurgeBundleWithCoordChildWithWFChild2
.For the complete list of flaky tests, see TEST-SUMMARY-FULL files.
{color:red}-1 DISTRO{color}
.{color:red}-1{color} distro tarball fails with the patch
{color:red}*-1 Overall result, please check the reported -1(s)*{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/PreCommit-OOZIE-Build/657/
> Improve sharelib upload to accept multiple source folders
> -
>
> Key: OOZIE-2829
> URL: https://issues.apache.org/jira/browse/OOZIE-2829
> Project: Oozie
> Issue Type: Bug
> Components: tools
>Affects Versions: 5.0.0
>Reporter: Peter Cseh
>Assignee: Julia Kinga Marton
>Priority: Major
> Fix For: 5.1.0
>
> Attachments: OOZIE-2829-006.patch, OOZIE-2829-01.patch,
> OOZIE-2829-02.patch, OOZIE-2829-03.patch, OOZIE-2829-04.patch,
> OOZIE-2829-05.patch
>
>
> Right now sharelib can be created via {{sharelib create -fs FS_URI -locallib
> SHARED_LIBRARY}} where the SHARED_LIBRARY can be a tarbal or a folder.
> It would be nice to have the possibility to define additional folders to be
> uploaded into the sharelib, so the users don't have to copy or link the files
> together on their machine.
> The syntax could be something like -additional-lib
> sharelibName=/path/to/source/;/path/to/some/file,sharelibName2=/path/to/some/folder
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-2829) Improve sharelib upload to accept multiple source folders
[
https://issues.apache.org/jira/browse/OOZIE-2829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16533612#comment-16533612
]
Hadoop QA commented on OOZIE-2829:
--
PreCommit-OOZIE-Build started
> Improve sharelib upload to accept multiple source folders
> -
>
> Key: OOZIE-2829
> URL: https://issues.apache.org/jira/browse/OOZIE-2829
> Project: Oozie
> Issue Type: Bug
> Components: tools
>Affects Versions: 5.0.0
>Reporter: Peter Cseh
>Assignee: Julia Kinga Marton
>Priority: Major
> Fix For: 5.1.0
>
> Attachments: OOZIE-2829-006.patch, OOZIE-2829-01.patch,
> OOZIE-2829-02.patch, OOZIE-2829-03.patch, OOZIE-2829-04.patch,
> OOZIE-2829-05.patch
>
>
> Right now sharelib can be created via {{sharelib create -fs FS_URI -locallib
> SHARED_LIBRARY}} where the SHARED_LIBRARY can be a tarbal or a folder.
> It would be nice to have the possibility to define additional folders to be
> uploaded into the sharelib, so the users don't have to copy or link the files
> together on their machine.
> The syntax could be something like -additional-lib
> sharelibName=/path/to/source/;/path/to/some/file,sharelibName2=/path/to/some/folder
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-2829) Improve sharelib upload to accept multiple source folders
[
https://issues.apache.org/jira/browse/OOZIE-2829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16516821#comment-16516821
]
Julia Kinga Marton commented on OOZIE-2829:
---
[~asasvari], I can take it over
> Improve sharelib upload to accept multiple source folders
> -
>
> Key: OOZIE-2829
> URL: https://issues.apache.org/jira/browse/OOZIE-2829
> Project: Oozie
> Issue Type: Bug
> Components: tools
>Reporter: Peter Cseh
>Assignee: Attila Sasvari
>Priority: Major
> Attachments: OOZIE-2829-01.patch, OOZIE-2829-02.patch,
> OOZIE-2829-03.patch, OOZIE-2829-04.patch, OOZIE-2829-05.patch
>
>
> Right now sharelib can be created via {{sharelib create -fs FS_URI -locallib
> SHARED_LIBRARY}} where the SHARED_LIBRARY can be a tarbal or a folder.
> It would be nice to have the possibility to define additional folders to be
> uploaded into the sharelib, so the users don't have to copy or link the files
> together on their machine.
> The syntax could be something like -additional-lib
> sharelibName=/path/to/source/;/path/to/some/file,sharelibName2=/path/to/some/folder
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-2829) Improve sharelib upload to accept multiple source folders
[
https://issues.apache.org/jira/browse/OOZIE-2829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16516813#comment-16516813
]
Attila Sasvari commented on OOZIE-2829:
---
[~andras.piros], [~pbacsko], [~kmarton] or anyone who reads it: I don't have to
work on this improvement. Feel free to take it over.
> Improve sharelib upload to accept multiple source folders
> -
>
> Key: OOZIE-2829
> URL: https://issues.apache.org/jira/browse/OOZIE-2829
> Project: Oozie
> Issue Type: Bug
> Components: tools
>Reporter: Peter Cseh
>Assignee: Attila Sasvari
>Priority: Major
> Attachments: OOZIE-2829-01.patch, OOZIE-2829-02.patch,
> OOZIE-2829-03.patch, OOZIE-2829-04.patch, OOZIE-2829-05.patch
>
>
> Right now sharelib can be created via {{sharelib create -fs FS_URI -locallib
> SHARED_LIBRARY}} where the SHARED_LIBRARY can be a tarbal or a folder.
> It would be nice to have the possibility to define additional folders to be
> uploaded into the sharelib, so the users don't have to copy or link the files
> together on their machine.
> The syntax could be something like -additional-lib
> sharelibName=/path/to/source/;/path/to/some/file,sharelibName2=/path/to/some/folder
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-2829) Improve sharelib upload to accept multiple source folders
[
https://issues.apache.org/jira/browse/OOZIE-2829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16515676#comment-16515676
]
Hadoop QA commented on OOZIE-2829:
--
Testing JIRA OOZIE-2829
Cleaning local git workspace
{color:red}-1{color} Patch failed to apply to head of branch
> Improve sharelib upload to accept multiple source folders
> -
>
> Key: OOZIE-2829
> URL: https://issues.apache.org/jira/browse/OOZIE-2829
> Project: Oozie
> Issue Type: Bug
> Components: tools
>Reporter: Peter Cseh
>Assignee: Attila Sasvari
>Priority: Major
> Attachments: OOZIE-2829-01.patch, OOZIE-2829-02.patch,
> OOZIE-2829-03.patch, OOZIE-2829-04.patch, OOZIE-2829-05.patch
>
>
> Right now sharelib can be created via {{sharelib create -fs FS_URI -locallib
> SHARED_LIBRARY}} where the SHARED_LIBRARY can be a tarbal or a folder.
> It would be nice to have the possibility to define additional folders to be
> uploaded into the sharelib, so the users don't have to copy or link the files
> together on their machine.
> The syntax could be something like -additional-lib
> sharelibName=/path/to/source/;/path/to/some/file,sharelibName2=/path/to/some/folder
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-2829) Improve sharelib upload to accept multiple source folders
[
https://issues.apache.org/jira/browse/OOZIE-2829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16515671#comment-16515671
]
Hadoop QA commented on OOZIE-2829:
--
PreCommit-OOZIE-Build started
> Improve sharelib upload to accept multiple source folders
> -
>
> Key: OOZIE-2829
> URL: https://issues.apache.org/jira/browse/OOZIE-2829
> Project: Oozie
> Issue Type: Bug
> Components: tools
>Reporter: Peter Cseh
>Assignee: Attila Sasvari
>Priority: Major
> Attachments: OOZIE-2829-01.patch, OOZIE-2829-02.patch,
> OOZIE-2829-03.patch, OOZIE-2829-04.patch, OOZIE-2829-05.patch
>
>
> Right now sharelib can be created via {{sharelib create -fs FS_URI -locallib
> SHARED_LIBRARY}} where the SHARED_LIBRARY can be a tarbal or a folder.
> It would be nice to have the possibility to define additional folders to be
> uploaded into the sharelib, so the users don't have to copy or link the files
> together on their machine.
> The syntax could be something like -additional-lib
> sharelibName=/path/to/source/;/path/to/some/file,sharelibName2=/path/to/some/folder
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-2829) Improve sharelib upload to accept multiple source folders
[
https://issues.apache.org/jira/browse/OOZIE-2829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16515661#comment-16515661
]
Hadoop QA commented on OOZIE-2829:
--
Testing JIRA OOZIE-2829
Cleaning local git workspace
{color:red}-1{color} Patch failed to apply to head of branch
> Improve sharelib upload to accept multiple source folders
> -
>
> Key: OOZIE-2829
> URL: https://issues.apache.org/jira/browse/OOZIE-2829
> Project: Oozie
> Issue Type: Bug
> Components: tools
>Reporter: Peter Cseh
>Assignee: Attila Sasvari
>Priority: Major
> Attachments: OOZIE-2829-01.patch, OOZIE-2829-02.patch,
> OOZIE-2829-03.patch, OOZIE-2829-04.patch, OOZIE-2829-05.patch
>
>
> Right now sharelib can be created via {{sharelib create -fs FS_URI -locallib
> SHARED_LIBRARY}} where the SHARED_LIBRARY can be a tarbal or a folder.
> It would be nice to have the possibility to define additional folders to be
> uploaded into the sharelib, so the users don't have to copy or link the files
> together on their machine.
> The syntax could be something like -additional-lib
> sharelibName=/path/to/source/;/path/to/some/file,sharelibName2=/path/to/some/folder
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-2829) Improve sharelib upload to accept multiple source folders
[
https://issues.apache.org/jira/browse/OOZIE-2829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16515658#comment-16515658
]
Hadoop QA commented on OOZIE-2829:
--
PreCommit-OOZIE-Build started
> Improve sharelib upload to accept multiple source folders
> -
>
> Key: OOZIE-2829
> URL: https://issues.apache.org/jira/browse/OOZIE-2829
> Project: Oozie
> Issue Type: Bug
> Components: tools
>Reporter: Peter Cseh
>Assignee: Attila Sasvari
>Priority: Major
> Attachments: OOZIE-2829-01.patch, OOZIE-2829-02.patch,
> OOZIE-2829-03.patch, OOZIE-2829-04.patch, OOZIE-2829-05.patch
>
>
> Right now sharelib can be created via {{sharelib create -fs FS_URI -locallib
> SHARED_LIBRARY}} where the SHARED_LIBRARY can be a tarbal or a folder.
> It would be nice to have the possibility to define additional folders to be
> uploaded into the sharelib, so the users don't have to copy or link the files
> together on their machine.
> The syntax could be something like -additional-lib
> sharelibName=/path/to/source/;/path/to/some/file,sharelibName2=/path/to/some/folder
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-2829) Improve sharelib upload to accept multiple source folders
[
https://issues.apache.org/jira/browse/OOZIE-2829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16515657#comment-16515657
]
Andras Piros commented on OOZIE-2829:
-
Let us see whether the latest patch can still be applied to current {{master}}.
> Improve sharelib upload to accept multiple source folders
> -
>
> Key: OOZIE-2829
> URL: https://issues.apache.org/jira/browse/OOZIE-2829
> Project: Oozie
> Issue Type: Bug
> Components: tools
>Reporter: Peter Cseh
>Assignee: Attila Sasvari
>Priority: Major
> Attachments: OOZIE-2829-01.patch, OOZIE-2829-02.patch,
> OOZIE-2829-03.patch, OOZIE-2829-04.patch, OOZIE-2829-05.patch
>
>
> Right now sharelib can be created via {{sharelib create -fs FS_URI -locallib
> SHARED_LIBRARY}} where the SHARED_LIBRARY can be a tarbal or a folder.
> It would be nice to have the possibility to define additional folders to be
> uploaded into the sharelib, so the users don't have to copy or link the files
> together on their machine.
> The syntax could be something like -additional-lib
> sharelibName=/path/to/source/;/path/to/some/file,sharelibName2=/path/to/some/folder
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-2829) Improve sharelib upload to accept multiple source folders
[
https://issues.apache.org/jira/browse/OOZIE-2829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16033705#comment-16033705
]
Hadoop QA commented on OOZIE-2829:
--
Testing JIRA OOZIE-2829
Cleaning local git workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than
132
.{color:green}+1{color} the patch does adds/modifies 7 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warnings
.{color:red}WARNING{color}: the current HEAD has 6 Javadoc warning(s)
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:green}+1{color} There are no new bugs found in total.
. {color:green}+1{color} There are no new bugs found in [server].
. {color:green}+1{color} There are no new bugs found in [client].
. {color:green}+1{color} There are no new bugs found in [core].
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:green}+1{color} There are no new bugs found in [tools].
. {color:green}+1{color} There are no new bugs found in [examples].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [sharelib/oozie].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
.Tests run: 1960
.Tests rerun: 79
.Tests failed at first run:
org.apache.oozie.action.hadoop.TestJavaActionExecutor,org.apache.oozie.jms.TestJMSJobEventListener,org.apache.oozie.action.hadoop.TestLauncherAM,
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch
{color:green}*+1 Overall result, good!, no -1s*{color}
{color:red}. There is at least one warning, please check{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/oozie-trunk-precommit-build/3855/
> Improve sharelib upload to accept multiple source folders
> -
>
> Key: OOZIE-2829
> URL: https://issues.apache.org/jira/browse/OOZIE-2829
> Project: Oozie
> Issue Type: Bug
> Components: tools
>Reporter: Peter Cseh
>Assignee: Attila Sasvari
> Attachments: OOZIE-2829-01.patch, OOZIE-2829-02.patch,
> OOZIE-2829-03.patch, OOZIE-2829-04.patch, OOZIE-2829-05.patch
>
>
> Right now sharelib can be created via {{sharelib create -fs FS_URI -locallib
> SHARED_LIBRARY}} where the SHARED_LIBRARY can be a tarbal or a folder.
> It would be nice to have the possibility to define additional folders to be
> uploaded into the sharelib, so the users don't have to copy or link the files
> together on their machine.
> The syntax could be something like -additional-lib
> sharelibName=/path/to/source/;/path/to/some/file,sharelibName2=/path/to/some/folder
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (OOZIE-2829) Improve sharelib upload to accept multiple source folders
[
https://issues.apache.org/jira/browse/OOZIE-2829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16033204#comment-16033204
]
Hadoop QA commented on OOZIE-2829:
--
Testing JIRA OOZIE-2829
Cleaning local git workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than
132
.{color:green}+1{color} the patch does adds/modifies 7 testcase(s)
{color:red}-1 RAT{color}
.{color:red}-1{color} the patch seems to introduce - new RAT warning(s)
.{color:red}WARNING: the current HEAD has RAT warning(s), they should be
addressed ASAP{color}
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warnings
{color:red}-1 COMPILE{color}
.{color:red}-1{color} HEAD does not compile
.{color:red}-1{color} patch does not compile
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:green}+1{color} There are no new bugs found in total.
. {color:green}+1{color} There are no new bugs found in [server].
. {color:green}+1{color} There are no new bugs found in [client].
. {color:green}+1{color} There are no new bugs found in [core].
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:green}+1{color} There are no new bugs found in [tools].
. {color:green}+1{color} There are no new bugs found in [examples].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [sharelib/oozie].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:red}-1 TESTS{color} - patch does not compile, cannot run testcases
{color:red}-1 DISTRO{color}
.{color:red}-1{color} distro tarball fails with the patch
{color:red}*-1 Overall result, please check the reported -1(s)*{color}
{color:red}. There is at least one warning, please check{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/oozie-trunk-precommit-build/3854/
> Improve sharelib upload to accept multiple source folders
> -
>
> Key: OOZIE-2829
> URL: https://issues.apache.org/jira/browse/OOZIE-2829
> Project: Oozie
> Issue Type: Bug
> Components: tools
>Reporter: Peter Cseh
>Assignee: Attila Sasvari
> Attachments: OOZIE-2829-01.patch, OOZIE-2829-02.patch,
> OOZIE-2829-03.patch, OOZIE-2829-04.patch, OOZIE-2829-05.patch
>
>
> Right now sharelib can be created via {{sharelib create -fs FS_URI -locallib
> SHARED_LIBRARY}} where the SHARED_LIBRARY can be a tarbal or a folder.
> It would be nice to have the possibility to define additional folders to be
> uploaded into the sharelib, so the users don't have to copy or link the files
> together on their machine.
> The syntax could be something like -additional-lib
> sharelibName=/path/to/source/;/path/to/some/file,sharelibName2=/path/to/some/folder
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (OOZIE-2829) Improve sharelib upload to accept multiple source folders
[
https://issues.apache.org/jira/browse/OOZIE-2829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16033031#comment-16033031
]
Hadoop QA commented on OOZIE-2829:
--
Testing JIRA OOZIE-2829
Cleaning local git workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than
132
.{color:green}+1{color} the patch does adds/modifies 7 testcase(s)
{color:red}-1 RAT{color}
.{color:red}-1{color} the patch seems to introduce - new RAT warning(s)
.{color:red}WARNING: the current HEAD has RAT warning(s), they should be
addressed ASAP{color}
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warnings
{color:red}-1 COMPILE{color}
.{color:red}-1{color} HEAD does not compile
.{color:red}-1{color} patch does not compile
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:green}+1{color} There are no new bugs found in total.
. {color:green}+1{color} There are no new bugs found in [server].
. {color:green}+1{color} There are no new bugs found in [client].
. {color:green}+1{color} There are no new bugs found in [core].
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:green}+1{color} There are no new bugs found in [tools].
. {color:green}+1{color} There are no new bugs found in [examples].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [sharelib/oozie].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:red}-1 TESTS{color} - patch does not compile, cannot run testcases
{color:red}-1 DISTRO{color}
.{color:red}-1{color} distro tarball fails with the patch
{color:red}*-1 Overall result, please check the reported -1(s)*{color}
{color:red}. There is at least one warning, please check{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/oozie-trunk-precommit-build/3853/
> Improve sharelib upload to accept multiple source folders
> -
>
> Key: OOZIE-2829
> URL: https://issues.apache.org/jira/browse/OOZIE-2829
> Project: Oozie
> Issue Type: Bug
> Components: tools
>Reporter: Peter Cseh
>Assignee: Attila Sasvari
> Attachments: OOZIE-2829-01.patch, OOZIE-2829-02.patch,
> OOZIE-2829-03.patch, OOZIE-2829-04.patch, OOZIE-2829-05.patch
>
>
> Right now sharelib can be created via {{sharelib create -fs FS_URI -locallib
> SHARED_LIBRARY}} where the SHARED_LIBRARY can be a tarbal or a folder.
> It would be nice to have the possibility to define additional folders to be
> uploaded into the sharelib, so the users don't have to copy or link the files
> together on their machine.
> The syntax could be something like -additional-lib
> sharelibName=/path/to/source/;/path/to/some/file,sharelibName2=/path/to/some/folder
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (OOZIE-2829) Improve sharelib upload to accept multiple source folders
[
https://issues.apache.org/jira/browse/OOZIE-2829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16032989#comment-16032989
]
Peter Bacsko commented on OOZIE-2829:
-
[~asasvari] please correct the compilation problem:
{noformat}
[INFO] Apache Oozie Tools . FAILURE [ 2.460 s]
[INFO] Apache Oozie MiniOozie . SKIPPED
[INFO] Apache Oozie Server SKIPPED
[INFO] Apache Oozie Distro SKIPPED
[INFO] Apache Oozie ZooKeeper Security Tests .. SKIPPED
[INFO]
[INFO] BUILD FAILURE
[INFO]
[INFO] Total time: 03:17 min
[INFO] Finished at: 2017-06-01T15:30:46+02:00
[INFO] Final Memory: 493M/1770M
[INFO]
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:2.3.2:testCompile
(default-testCompile) on project oozie-tools: Compilation failure
[ERROR]
/Users/pbacsko/oozie_commit/oozie/tools/src/test/java/org/apache/oozie/tools/TestOozieSharelibCLI.java:[132,27]
error: checkIfSourceFilesExist(File,Path) in cannot override
checkIfSourceFilesExist(File,Path) in OozieSharelibCLI
{noformat}
> Improve sharelib upload to accept multiple source folders
> -
>
> Key: OOZIE-2829
> URL: https://issues.apache.org/jira/browse/OOZIE-2829
> Project: Oozie
> Issue Type: Bug
> Components: tools
>Reporter: Peter Cseh
>Assignee: Attila Sasvari
> Attachments: OOZIE-2829-01.patch, OOZIE-2829-02.patch,
> OOZIE-2829-03.patch, OOZIE-2829-04.patch
>
>
> Right now sharelib can be created via {{sharelib create -fs FS_URI -locallib
> SHARED_LIBRARY}} where the SHARED_LIBRARY can be a tarbal or a folder.
> It would be nice to have the possibility to define additional folders to be
> uploaded into the sharelib, so the users don't have to copy or link the files
> together on their machine.
> The syntax could be something like -additional-lib
> sharelibName=/path/to/source/;/path/to/some/file,sharelibName2=/path/to/some/folder
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (OOZIE-2829) Improve sharelib upload to accept multiple source folders
[
https://issues.apache.org/jira/browse/OOZIE-2829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16032948#comment-16032948
]
Hadoop QA commented on OOZIE-2829:
--
Testing JIRA OOZIE-2829
Cleaning local git workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than
132
.{color:green}+1{color} the patch does adds/modifies 7 testcase(s)
{color:red}-1 RAT{color}
.{color:red}-1{color} the patch seems to introduce - new RAT warning(s)
.{color:red}WARNING: the current HEAD has RAT warning(s), they should be
addressed ASAP{color}
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warnings
{color:red}-1 COMPILE{color}
.{color:red}-1{color} HEAD does not compile
.{color:red}-1{color} patch does not compile
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:green}+1{color} There are no new bugs found in total.
. {color:green}+1{color} There are no new bugs found in [server].
. {color:green}+1{color} There are no new bugs found in [client].
. {color:green}+1{color} There are no new bugs found in [core].
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:green}+1{color} There are no new bugs found in [tools].
. {color:green}+1{color} There are no new bugs found in [examples].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [sharelib/oozie].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:red}-1 TESTS{color} - patch does not compile, cannot run testcases
{color:red}-1 DISTRO{color}
.{color:red}-1{color} distro tarball fails with the patch
{color:red}*-1 Overall result, please check the reported -1(s)*{color}
{color:red}. There is at least one warning, please check{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/oozie-trunk-precommit-build/3852/
> Improve sharelib upload to accept multiple source folders
> -
>
> Key: OOZIE-2829
> URL: https://issues.apache.org/jira/browse/OOZIE-2829
> Project: Oozie
> Issue Type: Bug
> Components: tools
>Reporter: Peter Cseh
>Assignee: Attila Sasvari
> Attachments: OOZIE-2829-01.patch, OOZIE-2829-02.patch,
> OOZIE-2829-03.patch, OOZIE-2829-04.patch
>
>
> Right now sharelib can be created via {{sharelib create -fs FS_URI -locallib
> SHARED_LIBRARY}} where the SHARED_LIBRARY can be a tarbal or a folder.
> It would be nice to have the possibility to define additional folders to be
> uploaded into the sharelib, so the users don't have to copy or link the files
> together on their machine.
> The syntax could be something like -additional-lib
> sharelibName=/path/to/source/;/path/to/some/file,sharelibName2=/path/to/some/folder
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (OOZIE-2829) Improve sharelib upload to accept multiple source folders
[
https://issues.apache.org/jira/browse/OOZIE-2829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16029491#comment-16029491
]
Hadoop QA commented on OOZIE-2829:
--
Testing JIRA OOZIE-2829
Cleaning local git workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than
132
.{color:green}+1{color} the patch does adds/modifies 7 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warnings
.{color:red}WARNING{color}: the current HEAD has 6 Javadoc warning(s)
{color:red}-1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:red}-1{color} patch does not compile
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:red}-1{color} There are [104] new bugs found below threshold in total
that must be fixed.
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
. {color:green}+1{color} There are no new bugs found in [sharelib/oozie].
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [server].
. {color:green}+1{color} There are no new bugs found in [core].
. {color:green}+1{color} There are no new bugs found in [tools].
. {color:red}-1{color} There are [98] new bugs found below threshold in [docs]
that must be fixed, listing only the first [5] ones.
. You can find the FindBugs diff here (look for the red and orange ones):
docs/findbugs-new.html
. The top [5] most important FindBugs errors are:
. At CLIParser.java:[line 169]: Found reliance on default encoding in
org.apache.oozie.cli.CLIParser.showHelp(CommandLine): new
java.io.PrintWriter(OutputStream)
. At CLIParser.java:[line 57]: new org.apache.oozie.cli.CLIParser(String,
String[]) may expose internal representation by storing an externally mutable
object into CLIParser.cliHelp
. At CLIParser.java:[lines 80-104]: Should
org.apache.oozie.cli.CLIParser$Command be a _static_ inner class?
. At OozieCLI.java:[line 845]: Found reliance on default encoding in
org.apache.oozie.cli.OozieCLI.getConfiguration(OozieClient, CommandLine): new
java.io.FileReader(File)
. At OozieCLI.java:[line 2171]: Found reliance on default encoding in
org.apache.oozie.cli.OozieCLI.validateCommandV41(CommandLine): new
java.io.FileReader(File)
. {color:red}-1{color} There are [6] new bugs found below threshold in [client]
that must be fixed, listing only the first [5] ones.
. You can find the FindBugs diff here (look for the red and orange ones):
client/findbugs-new.html
. The top [5] most important FindBugs errors are:
. At DateList.java:[line 55]: Nullcheck of date at line 55 of value previously
dereferenced in org.apache.oozie.example.DateList.main(String[])
. Private method org.apache.oozie.example.DateList.formatDateUTC(Calendar) is
never called: Redundant null check at DateList.java:[line 62]
. org.apache.oozie.example.LocalOozieExample.execute(String[]) may fail to
clean up java.io.InputStream: At DateList.java:[line 97]
. Path continues at LocalOozieExample.java:[line 76]: Obligation to clean up
resource created at LocalOozieExample.java:[line 72] is not discharged
. Path continues at LocalOozieExample.java:[line 78]: Path continues at
LocalOozieExample.java:[line 77]
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:red}-1 TESTS{color} - patch does not compile, cannot run testcases
{color:red}-1 DISTRO{color}
.{color:red}-1{color} distro tarball fails with the patch
{color:red}*-1 Overall result, please check the reported -1(s)*{color}
{color:red}. There is at least one warning, please check{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/oozie-trunk-precommit-build/3842/
> Improve sharelib upload
[jira] [Commented] (OOZIE-2829) Improve sharelib upload to accept multiple source folders
[
https://issues.apache.org/jira/browse/OOZIE-2829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16029306#comment-16029306
]
Attila Sasvari commented on OOZIE-2829:
---
[~pbacsko] thanks for the review.
Regarding {{SECURITY_MANAGER}} - thanks for pointing this out, it came from
the existing code. This naming issue is currently present in multiple classes:
--
https://github.com/apache/oozie/search?utf8=%E2%9C%93=SECURITY_MANAGER=
--
https://github.com/apache/oozie/blob/branch-4.3/tools/src/test/java/org/apache/oozie/tools/TestOozieSharelibCLI.java#L46
-- anyway I will fix these
I will attach a new patch addressing the issues above soon.
> Improve sharelib upload to accept multiple source folders
> -
>
> Key: OOZIE-2829
> URL: https://issues.apache.org/jira/browse/OOZIE-2829
> Project: Oozie
> Issue Type: Bug
> Components: tools
>Reporter: Peter Cseh
>Assignee: Attila Sasvari
> Attachments: OOZIE-2829-01.patch, OOZIE-2829-02.patch,
> OOZIE-2829-03.patch
>
>
> Right now sharelib can be created via {{sharelib create -fs FS_URI -locallib
> SHARED_LIBRARY}} where the SHARED_LIBRARY can be a tarbal or a folder.
> It would be nice to have the possibility to define additional folders to be
> uploaded into the sharelib, so the users don't have to copy or link the files
> together on their machine.
> The syntax could be something like -additional-lib
> sharelibName=/path/to/source/;/path/to/some/file,sharelibName2=/path/to/some/folder
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (OOZIE-2829) Improve sharelib upload to accept multiple source folders
[
https://issues.apache.org/jira/browse/OOZIE-2829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15996719#comment-15996719
]
Peter Bacsko commented on OOZIE-2829:
-
Couple of comments:
- the argument {{-additional_libs}} looks a bit weird to me because of the "_",
isn't there a better name for this? I know it's just nitpicking, but still...
:D Like {{-extralibs}} ?
- the method name {{checkSourceFilesExistance()}} sounds weird too, how about
{{checkIfSourceFilesExist()}}
- couple of protected methods, but only one of them is marked with
@VisibleForTesting - is it necessary to declare them protected?
- throw {{FileNotFoundException}} if a file does not exist (currently
{{IOException}} is thrown)
- Throw a more specific exception here as well (like IllegalArgumentException):
{{throw new Exception(String.format("Additional sharelib, '%s', has been
specified multiple times}}
- unnecessary comment: {{Integration tests for OozieSharelibCLI}}
- {{SECURITY_MANAGER}} it's not constant, shouldn't be full uppercase
After you addressed these, could you pls create an RB review? It's better to
inspect this change there.
> Improve sharelib upload to accept multiple source folders
> -
>
> Key: OOZIE-2829
> URL: https://issues.apache.org/jira/browse/OOZIE-2829
> Project: Oozie
> Issue Type: Bug
> Components: tools
>Reporter: Peter Cseh
>Assignee: Attila Sasvari
> Attachments: OOZIE-2829-01.patch, OOZIE-2829-02.patch,
> OOZIE-2829-03.patch
>
>
> Right now sharelib can be created via {{sharelib create -fs FS_URI -locallib
> SHARED_LIBRARY}} where the SHARED_LIBRARY can be a tarbal or a folder.
> It would be nice to have the possibility to define additional folders to be
> uploaded into the sharelib, so the users don't have to copy or link the files
> together on their machine.
> The syntax could be something like -additional-lib
> sharelibName=/path/to/source/;/path/to/some/file,sharelibName2=/path/to/some/folder
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (OOZIE-2829) Improve sharelib upload to accept multiple source folders
[
https://issues.apache.org/jira/browse/OOZIE-2829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15987034#comment-15987034
]
Hadoop QA commented on OOZIE-2829:
--
Testing JIRA OOZIE-2829
Cleaning local git workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than
132
.{color:green}+1{color} the patch does adds/modifies 2 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warnings
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:green}+1{color} There are no new bugs found in total.
. {color:green}+1{color} There are no new bugs found in [core].
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [sharelib/oozie].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [server].
. {color:green}+1{color} There are no new bugs found in [client].
. {color:green}+1{color} There are no new bugs found in
[hadooplibs/hadoop-utils-2].
. {color:green}+1{color} There are no new bugs found in [tools].
. {color:green}+1{color} There are no new bugs found in [examples].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:red}-1 TESTS{color}
.Tests run: 1933
.Tests failed: 1
.Tests errors: 2
.The patch failed the following testcases:
. testNofindLogs(org.apache.oozie.util.TestTimestampedMessageParser)
.Tests failing with errors:
.
testProcessRemainingLog(org.apache.oozie.util.TestTimestampedMessageParser)
.
testProcessRemainingCoordinatorLogForActions(org.apache.oozie.util.TestTimestampedMessageParser)
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch
{color:red}*-1 Overall result, please check the reported -1(s)*{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/oozie-trunk-precommit-build/3795/
> Improve sharelib upload to accept multiple source folders
> -
>
> Key: OOZIE-2829
> URL: https://issues.apache.org/jira/browse/OOZIE-2829
> Project: Oozie
> Issue Type: Bug
> Components: tools
>Reporter: Peter Cseh
>Assignee: Attila Sasvari
> Attachments: OOZIE-2829-01.patch, OOZIE-2829-02.patch,
> OOZIE-2829-03.patch
>
>
> Right now sharelib can be created via {{sharelib create -fs FS_URI -locallib
> SHARED_LIBRARY}} where the SHARED_LIBRARY can be a tarbal or a folder.
> It would be nice to have the possibility to define additional folders to be
> uploaded into the sharelib, so the users don't have to copy or link the files
> together on their machine.
> The syntax could be something like -additional-lib
> sharelibName=/path/to/source/;/path/to/some/file,sharelibName2=/path/to/some/folder
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (OOZIE-2829) Improve sharelib upload to accept multiple source folders
[
https://issues.apache.org/jira/browse/OOZIE-2829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15986331#comment-15986331
]
Attila Sasvari commented on OOZIE-2829:
---
It might make sense to finish, review, fix, commit OOZIE-2629 and OOZIE-2791
prior to this.
> Improve sharelib upload to accept multiple source folders
> -
>
> Key: OOZIE-2829
> URL: https://issues.apache.org/jira/browse/OOZIE-2829
> Project: Oozie
> Issue Type: Bug
> Components: tools
>Reporter: Peter Cseh
>Assignee: Attila Sasvari
> Attachments: OOZIE-2829-01.patch, OOZIE-2829-02.patch
>
>
> Right now sharelib can be created via {{sharelib create -fs FS_URI -locallib
> SHARED_LIBRARY}} where the SHARED_LIBRARY can be a tarbal or a folder.
> It would be nice to have the possibility to define additional folders to be
> uploaded into the sharelib, so the users don't have to copy or link the files
> together on their machine.
> The syntax could be something like -additional-lib
> sharelibName=/path/to/source/;/path/to/some/file,sharelibName2=/path/to/some/folder
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (OOZIE-2829) Improve sharelib upload to accept multiple source folders
[
https://issues.apache.org/jira/browse/OOZIE-2829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15986320#comment-15986320
]
Attila Sasvari commented on OOZIE-2829:
---
Re-executed tests in TestTimestampedMessageParser and each of them passed.
Attaching new patch that fixes new findbugs error soon.
> Improve sharelib upload to accept multiple source folders
> -
>
> Key: OOZIE-2829
> URL: https://issues.apache.org/jira/browse/OOZIE-2829
> Project: Oozie
> Issue Type: Bug
> Components: tools
>Reporter: Peter Cseh
>Assignee: Attila Sasvari
> Attachments: OOZIE-2829-01.patch, OOZIE-2829-02.patch
>
>
> Right now sharelib can be created via {{sharelib create -fs FS_URI -locallib
> SHARED_LIBRARY}} where the SHARED_LIBRARY can be a tarbal or a folder.
> It would be nice to have the possibility to define additional folders to be
> uploaded into the sharelib, so the users don't have to copy or link the files
> together on their machine.
> The syntax could be something like -additional-lib
> sharelibName=/path/to/source/;/path/to/some/file,sharelibName2=/path/to/some/folder
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (OOZIE-2829) Improve sharelib upload to accept multiple source folders
[
https://issues.apache.org/jira/browse/OOZIE-2829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15985330#comment-15985330
]
Hadoop QA commented on OOZIE-2829:
--
Testing JIRA OOZIE-2829
Cleaning local git workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any line longer than
132
.{color:green}+1{color} the patch does adds/modifies 2 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warnings
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:red}-1{color} There are [1] new bugs found below threshold in total that
must be fixed.
. {color:green}+1{color} There are no new bugs found in [server].
. {color:green}+1{color} There are no new bugs found in [client].
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [sharelib/oozie].
. {color:green}+1{color} There are no new bugs found in
[hadooplibs/hadoop-utils-2].
. {color:green}+1{color} There are no new bugs found in [core].
. {color:red}-1{color} There are [1] new bugs found below threshold in [tools]
that must be fixed.
. You can find the FindBugs diff here (look for the red and orange ones):
tools/findbugs-new.html
. The most important FindBugs errors are:
. At OozieSharelibCLI.java:[line 169]:
org.apache.oozie.tools.OozieSharelibCLI.run(String[]) invokes System.exit(...),
which shuts down the entire virtual machine
. {color:green}+1{color} There are no new bugs found in [examples].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:red}-1 TESTS{color}
.Tests run: 1933
.Tests failed: 1
.Tests errors: 2
.The patch failed the following testcases:
. testNofindLogs(org.apache.oozie.util.TestTimestampedMessageParser)
.Tests failing with errors:
.
testProcessRemainingLog(org.apache.oozie.util.TestTimestampedMessageParser)
.
testProcessRemainingCoordinatorLogForActions(org.apache.oozie.util.TestTimestampedMessageParser)
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch
{color:red}*-1 Overall result, please check the reported -1(s)*{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/oozie-trunk-precommit-build/3794/
> Improve sharelib upload to accept multiple source folders
> -
>
> Key: OOZIE-2829
> URL: https://issues.apache.org/jira/browse/OOZIE-2829
> Project: Oozie
> Issue Type: Bug
> Components: tools
>Reporter: Peter Cseh
>Assignee: Attila Sasvari
> Attachments: OOZIE-2829-01.patch, OOZIE-2829-02.patch
>
>
> Right now sharelib can be created via {{sharelib create -fs FS_URI -locallib
> SHARED_LIBRARY}} where the SHARED_LIBRARY can be a tarbal or a folder.
> It would be nice to have the possibility to define additional folders to be
> uploaded into the sharelib, so the users don't have to copy or link the files
> together on their machine.
> The syntax could be something like -additional-lib
> sharelibName=/path/to/source/;/path/to/some/file,sharelibName2=/path/to/some/folder
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (OOZIE-2829) Improve sharelib upload to accept multiple source folders
[
https://issues.apache.org/jira/browse/OOZIE-2829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15981110#comment-15981110
]
Attila Sasvari commented on OOZIE-2829:
---
[~gezapeti] thanks for the review comments.
1. fixed
2. fixed - also added more tests to test invalid inputs. I also extracted two
existing integration tests to a new test class. I wanted to use JUnit 4
annotations to check exception is thrown when the input is invalid.
3. OK
4. I adjusted help messages of CLI command to include more details (also
increased width in CLIParser):
{noformat}
usage:
OozieSharelibCLI creates or upgrade sharelib for oozie
oozie-setup.sh create : create a new timestamped version of
oozie sharelib
-additional_libsAdditional sharelib
resources. This option requires a colon-separated list of pairs of
sharelibname and
coma-sperated list of pathnames in the following format:
"sharelib_name=pathname[,pathname...][,sharelib_name2=pathname[,pathname...]...]"
Caveats:
* Each pathname is either
a directory or a regular file (compressed files are not
extracted prior to the
upload operation).
* Sharelibnames shall be
specified only once.
* Do not upload multiple
conflicting library versions for an additional sharelib
directory as it may cause
runtime issues.
Example command:
$ oozie-setup.sh sharelib
create -fs hdfs://localhost:9000 -locallib \
oozie-sharelib.tar.gz
-additional_libs share2=dir2,file2:share3=file3
-concurrencyNumber of threads to be
used for copy operations. (default=1)
-fs URI of the fileSystem to
create oozie share library
-locallib Local share library
directory
{noformat}
5. I adjusted help messages of CLI command (see previous)
> Improve sharelib upload to accept multiple source folders
> -
>
> Key: OOZIE-2829
> URL: https://issues.apache.org/jira/browse/OOZIE-2829
> Project: Oozie
> Issue Type: Bug
> Components: tools
>Reporter: Peter Cseh
>Assignee: Attila Sasvari
> Attachments: OOZIE-2829-01.patch
>
>
> Right now sharelib can be created via {{sharelib create -fs FS_URI -locallib
> SHARED_LIBRARY}} where the SHARED_LIBRARY can be a tarbal or a folder.
> It would be nice to have the possibility to define additional folders to be
> uploaded into the sharelib, so the users don't have to copy or link the files
> together on their machine.
> The syntax could be something like -additional-lib
> sharelibName=/path/to/source/;/path/to/some/file,sharelibName2=/path/to/some/folder
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (OOZIE-2829) Improve sharelib upload to accept multiple source folders
[
https://issues.apache.org/jira/browse/OOZIE-2829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15978639#comment-15978639
]
Peter Cseh commented on OOZIE-2829:
---
{quote}
using semicolon as separator might not be a good a idea in Linux. It can work
if you surround option args with double quotes, but probably it might better to
use colon as sharelibname separator.
{quote}
We should use a syntax that makes our life easier. I'm okay with using colons
as separators.
{quote}
I am also not sure that we want to allow adding individual files instead of
directories when specifying the sources.
{quote}
I like the idea to allow files in that list as it is providing more flexibility.
Regarding the patch:
# I would call {{copyAdditionalLibs}} all the time as iterating over an empty
set is not a problem
# Please check the format in {{getAdditionalLibs}} more strictly (e.g {{if
(addLibParts.lenght != 2) }} raise an exception, do not allow defining the same
sharelib twice.)
# I think it would make sense to use one thread pool for every copy operation.
# Please define the format of the additional libs part either in the help text
of the option or on a wiki page
# As we're not accepting and extracting tar files in the additional_libs, this
should be mentioned in the documentation
> Improve sharelib upload to accept multiple source folders
> -
>
> Key: OOZIE-2829
> URL: https://issues.apache.org/jira/browse/OOZIE-2829
> Project: Oozie
> Issue Type: Bug
> Components: tools
>Reporter: Peter Cseh
>Assignee: Attila Sasvari
> Attachments: OOZIE-2829-01.patch
>
>
> Right now sharelib can be created via {{sharelib create -fs FS_URI -locallib
> SHARED_LIBRARY}} where the SHARED_LIBRARY can be a tarbal or a folder.
> It would be nice to have the possibility to define additional folders to be
> uploaded into the sharelib, so the users don't have to copy or link the files
> together on their machine.
> The syntax could be something like -additional-lib
> sharelibName=/path/to/source/;/path/to/some/file,sharelibName2=/path/to/some/folder
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (OOZIE-2829) Improve sharelib upload to accept multiple source folders
[
https://issues.apache.org/jira/browse/OOZIE-2829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15976377#comment-15976377
]
Attila Sasvari commented on OOZIE-2829:
---
[~gezapeti] could you take a look? Thanks!
> Improve sharelib upload to accept multiple source folders
> -
>
> Key: OOZIE-2829
> URL: https://issues.apache.org/jira/browse/OOZIE-2829
> Project: Oozie
> Issue Type: Bug
> Components: tools
>Reporter: Peter Cseh
>Assignee: Attila Sasvari
> Attachments: OOZIE-2829-01.patch
>
>
> Right now sharelib can be created via {{sharelib create -fs FS_URI -locallib
> SHARED_LIBRARY}} where the SHARED_LIBRARY can be a tarbal or a folder.
> It would be nice to have the possibility to define additional folders to be
> uploaded into the sharelib, so the users don't have to copy or link the files
> together on their machine.
> The syntax could be something like -additional-lib
> sharelibName=/path/to/source/;/path/to/some/file,sharelibName2=/path/to/some/folder
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (OOZIE-2829) Improve sharelib upload to accept multiple source folders
[
https://issues.apache.org/jira/browse/OOZIE-2829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15972573#comment-15972573
]
Attila Sasvari commented on OOZIE-2829:
---
[~gezapeti] using semicolon as separator might not be a good a idea in Linux.
It can work if you surround option args with double quotes, but probably it
might better to use colon as sharelibname separator. I am also not sure that we
want to allow adding individual files instead of directories when specifying
the sources.
> Improve sharelib upload to accept multiple source folders
> -
>
> Key: OOZIE-2829
> URL: https://issues.apache.org/jira/browse/OOZIE-2829
> Project: Oozie
> Issue Type: Bug
> Components: tools
>Reporter: Peter Cseh
>Assignee: Attila Sasvari
>
> Right now sharelib can be created via {{sharelib create -fs FS_URI -locallib
> SHARED_LIBRARY}} where the SHARED_LIBRARY can be a tarbal or a folder.
> It would be nice to have the possibility to define additional folders to be
> uploaded into the sharelib, so the users don't have to copy or link the files
> together on their machine.
> The syntax could be something like -additional-lib
> sharelibName=/path/to/source/;/path/to/some/file,sharelibName2=/path/to/some/folder
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (OOZIE-2829) Improve sharelib upload to accept multiple source folders
[
https://issues.apache.org/jira/browse/OOZIE-2829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15972487#comment-15972487
]
Attila Sasvari commented on OOZIE-2829:
---
{{org.apache.commons.cli.Option}} does not allow "-" in option => use
{{additional_libs}}.
Instead of {{-additional-lib
sharelibName=/path/to/source/;/path/to/some/file,sharelibName2=/path/to/some/folder}},
I would use {{-additional-lib
sharelibName=/path/to/source/,/path/to/some/file;sharelibName2=/path/to/some/folder}}
> Improve sharelib upload to accept multiple source folders
> -
>
> Key: OOZIE-2829
> URL: https://issues.apache.org/jira/browse/OOZIE-2829
> Project: Oozie
> Issue Type: Bug
> Components: tools
>Reporter: Peter Cseh
>Assignee: Attila Sasvari
>
> Right now sharelib can be created via {{sharelib create -fs FS_URI -locallib
> SHARED_LIBRARY}} where the SHARED_LIBRARY can be a tarbal or a folder.
> It would be nice to have the possibility to define additional folders to be
> uploaded into the sharelib, so the users don't have to copy or link the files
> together on their machine.
> The syntax could be something like -additional-lib
> sharelibName=/path/to/source/;/path/to/some/file,sharelibName2=/path/to/some/folder
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
