Re: [PR] more validation for containerName and blobKey to avoid access escape [jclouds]

2024-04-14 Thread via GitHub 


jixinchi commented on code in PR #203:
URL: https://github.com/apache/jclouds/pull/203#discussion_r1565125481


##
apis/filesystem/src/main/java/org/jclouds/filesystem/predicates/validators/internal/FilesystemBlobKeyValidatorImpl.java:
##
@@ -38,6 +38,8 @@ public void validate(String name) throws 
IllegalArgumentException {
 //blobkey cannot start with / (or \ in Windows) character
 if (name.startsWith("\\") || name.startsWith("/"))
 throw new IllegalArgumentException("Blob key '" + name + "' cannot 
start with \\ or /");
+if (name.contains("../"))
+throw new IllegalArgumentException("Blob key '" + name + "' cannot 
contain ../");

Review Comment:
   I think `..foo` should be allowed. I tried several object storage services, 
such as gcloud and aliyun oss, they all allow `..foo` as object key.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscr...@jclouds.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: [PR] more validation for containerName and blobKey to avoid access escape [jclouds]

2024-04-14 Thread via GitHub 


jixinchi commented on code in PR #203:
URL: https://github.com/apache/jclouds/pull/203#discussion_r1565116204


##
apis/filesystem/src/main/java/org/jclouds/filesystem/strategy/internal/FilesystemStorageStrategyImpl.java:
##
@@ -691,6 +698,8 @@ public BlobAccess getBlobAccess(String containerName, 
String blobName) {
 
@Override
public void setBlobAccess(String container, String name, BlobAccess access) 
{
+  filesystemContainerNameValidator.validate(container);
+  filesystemBlobKeyValidator.validate(name);

Review Comment:
   These methods have been applied by the validators. I just added some missing 
ones.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscr...@jclouds.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Build failed in Jenkins: JClouds » jclouds-with-credentials » jclouds-google-compute-engine-live-tests #190

2024-04-14 Thread Apache Jenkins Server 
See 


Changes:


--
Started by timer
Running as SYSTEM
No JDK named ‘JDK 1.8 (latest)’ found
[EnvInject] - Loading node environment variables.
Building remotely on builds43 (ubuntu) in workspace 

No JDK named ‘JDK 1.8 (latest)’ found
ERROR: A Maven installation needs to be available for this project to be 
built.Either your server has no Maven installations defined, or the requested 
Maven version does not exist.
No JDK named ‘JDK 1.8 (latest)’ found
No JDK named ‘JDK 1.8 (latest)’ found

Re: [PR] JCLOUDS-1629: Add COOL and COLD to Tier [jclouds]

2024-04-14 Thread via GitHub 


gaul commented on PR #204:
URL: https://github.com/apache/jclouds/pull/204#issuecomment-2053984948

   @timuralp what do you think?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscr...@jclouds.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[jira] [Created] (JCLOUDS-1635) Add another blobstore Tier

2024-04-14 Thread Andrew Gaul (Jira) 
Andrew Gaul created JCLOUDS-1635:


 Summary: Add another blobstore Tier
 Key: JCLOUDS-1635
 URL: https://issues.apache.org/jira/browse/JCLOUDS-1635
 Project: jclouds
  Issue Type: New Feature
  Components: jclouds-blobstore
Affects Versions: 2.6.0
Reporter: Andrew Gaul


Currently jclouds supports STANDARD, INFREQUENT, and ARCHIVE.  But AzureBlob, 
GCS, and S3 all support at least 4 tiers.  So let's add something cheaper than 
INFREQUENT but faster than ARCHIVE.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Resolved] (JCLOUDS-1634) Add AzureBlob cold access tier

2024-04-14 Thread Andrew Gaul (Jira) 


 [ 
https://issues.apache.org/jira/browse/JCLOUDS-1634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Gaul resolved JCLOUDS-1634.
--
Fix Version/s: 2.6.1
   Resolution: Fixed

> Add AzureBlob cold access tier
> --
>
> Key: JCLOUDS-1634
> URL: https://issues.apache.org/jira/browse/JCLOUDS-1634
> Project: jclouds
>  Issue Type: Improvement
>  Components: jclouds-blobstore
>Affects Versions: 2.6.0
>Reporter: Andrew Gaul
>Assignee: Andrew Gaul
>Priority: Minor
>  Labels: azureblob
> Fix For: 2.6.1
>
>
> Currently jclouds only supports hot, cool, and archive: 
> https://learn.microsoft.com/en-us/azure/storage/blobs/access-tiers-overview



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (JCLOUDS-1634) Add AzureBlob cold access tier

2024-04-14 Thread ASF subversion and git services (Jira) 


[ 
https://issues.apache.org/jira/browse/JCLOUDS-1634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17836911#comment-17836911
 ] 

ASF subversion and git services commented on JCLOUDS-1634:
--

Commit e1f34bbfa77619871fdf1af9b3090c8ff695869d in jclouds's branch 
refs/heads/master from Andrew Gaul
[ https://gitbox.apache.org/repos/asf?p=jclouds.git;h=e1f34bbfa7 ]

JCLOUDS-1634: Add AzureBlob COLD access tier

References gaul/s3proxy#625.


> Add AzureBlob cold access tier
> --
>
> Key: JCLOUDS-1634
> URL: https://issues.apache.org/jira/browse/JCLOUDS-1634
> Project: jclouds
>  Issue Type: Improvement
>  Components: jclouds-blobstore
>Affects Versions: 2.6.0
>Reporter: Andrew Gaul
>Assignee: Andrew Gaul
>Priority: Minor
>  Labels: azureblob
>
> Currently jclouds only supports hot, cool, and archive: 
> https://learn.microsoft.com/en-us/azure/storage/blobs/access-tiers-overview



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (JCLOUDS-1634) Add AzureBlob cold access tier

2024-04-14 Thread Andrew Gaul (Jira) 


 [ 
https://issues.apache.org/jira/browse/JCLOUDS-1634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Gaul updated JCLOUDS-1634:
-
Labels: azureblob  (was: )

> Add AzureBlob cold access tier
> --
>
> Key: JCLOUDS-1634
> URL: https://issues.apache.org/jira/browse/JCLOUDS-1634
> Project: jclouds
>  Issue Type: Improvement
>  Components: jclouds-blobstore
>Affects Versions: 2.6.0
>Reporter: Andrew Gaul
>Assignee: Andrew Gaul
>Priority: Minor
>  Labels: azureblob
>
> Currently jclouds only supports hot, cool, and archive: 
> https://learn.microsoft.com/en-us/azure/storage/blobs/access-tiers-overview



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (JCLOUDS-1634) Add AzureBlob cold access tier

2024-04-14 Thread Andrew Gaul (Jira) 
Andrew Gaul created JCLOUDS-1634:


 Summary: Add AzureBlob cold access tier
 Key: JCLOUDS-1634
 URL: https://issues.apache.org/jira/browse/JCLOUDS-1634
 Project: jclouds
  Issue Type: Improvement
  Components: jclouds-blobstore
Affects Versions: 2.6.0
Reporter: Andrew Gaul
Assignee: Andrew Gaul


Currently jclouds only supports hot, cool, and archive: 
https://learn.microsoft.com/en-us/azure/storage/blobs/access-tiers-overview



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Build failed in Jenkins: JClouds » jclouds #108

2024-04-14 Thread Apache Jenkins Server 
See 


Changes:

[Andrew Gaul] JCLOUDS-1631: fix AWSRequestAuthorizeSignatureV4 when prefix 
contains

[Andrew Gaul] add license header


--
Started by an SCM change
Running as SYSTEM
[EnvInject] - Loading node environment variables.
Building remotely on builds26 (ubuntu) in workspace 

The recommended git tool is: NONE
No credentials specified
Cloning the remote Git repository
Cloning repository https://gitbox.apache.org/repos/asf/jclouds.git
 > git init  # 
 > timeout=10
Fetching upstream changes from https://gitbox.apache.org/repos/asf/jclouds.git
 > git --version # timeout=10
 > git --version # 'git version 2.34.1'
 > git fetch --tags --force --progress -- 
 > https://gitbox.apache.org/repos/asf/jclouds.git 
 > +refs/heads/*:refs/remotes/origin/* # timeout=10
 > git config remote.origin.url https://gitbox.apache.org/repos/asf/jclouds.git 
 > # timeout=10
 > git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # 
 > timeout=10
Avoid second fetch
 > git rev-parse refs/remotes/origin/master^{commit} # timeout=10
Checking out Revision 4435765f550bbbae034a27ab5c5deee958fbc55f 
(refs/remotes/origin/master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 4435765f550bbbae034a27ab5c5deee958fbc55f # timeout=10
Commit message: "add license header"
 > git rev-list --no-walk bc43572d657956e013b094dfa08618c61a795e3d # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the 
SCM step.
[EnvInject] - Injecting as environment variables the properties content 
LC_ALL=en_US.UTF-8
LANG=en_US.UTF-8

[EnvInject] - Variables injected successfully.
Parsing POMs
Modules changed, recalculating dependency graph
Established TCP socket on 35203
maven35-agent.jar already up to date
maven35-interceptor.jar already up to date
maven3-interceptor-commons.jar already up to date
[jclouds] $ /home/jenkins/tools/java/latest1.8/bin/java -cp 
/home/jenkins/jenkins-agent/maven35-agent.jar:/home/jenkins/tools/maven/apache-maven-3.5.4/boot/plexus-classworlds-2.5.2.jar:/home/jenkins/tools/maven/apache-maven-3.5.4/conf/logging
 jenkins.maven3.agent.Maven35Main /home/jenkins/tools/maven/apache-maven-3.5.4 
/home/jenkins/jenkins-agent/agent.jar 
/home/jenkins/jenkins-agent/maven35-interceptor.jar 
/home/jenkins/jenkins-agent/maven3-interceptor-commons.jar 35203
Exception in thread "main" java.lang.UnsupportedClassVersionError: 
hudson/remoting/Launcher has been compiled by a more recent version of the Java 
Runtime (class file version 55.0), this version of the Java Runtime only 
recognizes class file versions up to 52.0
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:756)
at 
java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:473)
at java.net.URLClassLoader.access$100(URLClassLoader.java:74)
at java.net.URLClassLoader$1.run(URLClassLoader.java:369)
at java.net.URLClassLoader$1.run(URLClassLoader.java:363)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:362)
at 
org.codehaus.plexus.classworlds.realm.ClassRealm.loadClassFromSelf(ClassRealm.java:401)
at 
org.codehaus.plexus.classworlds.strategy.SelfFirstStrategy.loadClass(SelfFirstStrategy.java:42)
at 
org.codehaus.plexus.classworlds.realm.ClassRealm.unsynchronizedLoadClass(ClassRealm.java:271)
at 
org.codehaus.plexus.classworlds.realm.ClassRealm.loadClass(ClassRealm.java:247)
at 
org.codehaus.plexus.classworlds.realm.ClassRealm.loadClass(ClassRealm.java:239)
at jenkins.maven3.agent.Maven35Main.main(Maven35Main.java:136)
at jenkins.maven3.agent.Maven35Main.main(Maven35Main.java:66)
ERROR: Failed to parse POMs
java.io.EOFException: unexpected stream termination
at hudson.remoting.ChannelBuilder.negotiate(ChannelBuilder.java:459)
at hudson.remoting.ChannelBuilder.build(ChannelBuilder.java:404)
at hudson.slaves.Channels.forProcess(Channels.java:121)
at 
hudson.maven.AbstractMavenProcessFactory.newProcess(AbstractMavenProcessFactory.java:298)
at hudson.maven.ProcessCache.get(ProcessCache.java:237)
at 
hudson.maven.MavenModuleSetBuild$MavenModuleSetBuildExecution.doRun(MavenModuleSetBuild.java:802)
at 
hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:526)
at hudson.model.Run.execute(Run.java:1841)
at hudson.maven.MavenModuleSetBuild.run(MavenModuleSetBuild.java:543)
at hudson.model.ResourceController.execute(ResourceController.java:101)
at

Re: [PR] JCLOUDS-1631: fix AWSRequestAuthorizeSignatureV4 when query contains special chars [jclouds]

2024-04-14 Thread via GitHub 


gaul merged PR #200:
URL: https://github.com/apache/jclouds/pull/200


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscr...@jclouds.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: [PR] JCLOUDS-1631: fix AWSRequestAuthorizeSignatureV4 when query contains special chars [jclouds]

2024-04-14 Thread via GitHub 


gaul commented on PR #200:
URL: https://github.com/apache/jclouds/pull/200#issuecomment-2053967281

   Thank you for your contribution @Maxim-Gadalov!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscr...@jclouds.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[jira] [Resolved] (JCLOUDS-1631) AWS S3, sign for authorization header failed if query part contains special chars

2024-04-14 Thread Andrew Gaul (Jira) 


 [ 
https://issues.apache.org/jira/browse/JCLOUDS-1631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Gaul resolved JCLOUDS-1631.
--
Fix Version/s: 2.6.1
 Assignee: Andrew Gaul
   Resolution: Fixed

> AWS S3, sign for authorization header failed if query part contains special 
> chars
> -
>
> Key: JCLOUDS-1631
> URL: https://issues.apache.org/jira/browse/JCLOUDS-1631
> Project: jclouds
>  Issue Type: Bug
>  Components: jclouds-blobstore
>Affects Versions: 2.5.0
>Reporter: Maksim Hadalau
>Assignee: Andrew Gaul
>Priority: Major
> Fix For: 2.6.1
>
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> Problem description:
> Can not list blobs for aws-s3 provider if prefix contains special chars %/&
> Steps to reproduce:
> try to list blobs with following prefix: 
> "Folder (`~!@#$%^&*-_+[]'|<>.?) Name/"
> Actual behavior: 
> Error: URLDecoder: Incomplete trailing escape (%) pattern
> Expected behavior:
> provided prefix must be listed
> Problem location:
> `AWSRequestAuthorizeSignatureV4.signForAuthorizationHeader()`
>  
> Multimap queryMap = 
> queryParser().apply(request.getEndpoint().getQuery());
> request.getEndpoint().getQuery() - returns a decoded query string
> however queryParser() require encoded one
> Fix:
> Multimap queryMap = 
> queryParser().apply(request.getEndpoint().getRawQuery());
> When jclouds generates a request to the AWS it encodes prefix (encoding all 
> special chars in it, including % and &), however calling `getQuery()` returns 
> decoded version of query string which lead to unpredictable behavior. 
> P.S. required patch in 2.5.x if possible
> PR https://github.com/apache/jclouds/pull/200



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (JCLOUDS-1631) AWS S3, sign for authorization header failed if query part contains special chars

2024-04-14 Thread ASF subversion and git services (Jira) 


[ 
https://issues.apache.org/jira/browse/JCLOUDS-1631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17836909#comment-17836909
 ] 

ASF subversion and git services commented on JCLOUDS-1631:
--

Commit 0688553087aabdaa7f503265bbc4a4cb60adef55 in jclouds's branch 
refs/heads/master from Maksim_Hadalau
[ https://gitbox.apache.org/repos/asf?p=jclouds.git;h=0688553087 ]

JCLOUDS-1631: fix AWSRequestAuthorizeSignatureV4 when prefix contains special 
chars


> AWS S3, sign for authorization header failed if query part contains special 
> chars
> -
>
> Key: JCLOUDS-1631
> URL: https://issues.apache.org/jira/browse/JCLOUDS-1631
> Project: jclouds
>  Issue Type: Bug
>  Components: jclouds-blobstore
>Affects Versions: 2.5.0
>Reporter: Maksim Hadalau
>Priority: Major
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> Problem description:
> Can not list blobs for aws-s3 provider if prefix contains special chars %/&
> Steps to reproduce:
> try to list blobs with following prefix: 
> "Folder (`~!@#$%^&*-_+[]'|<>.?) Name/"
> Actual behavior: 
> Error: URLDecoder: Incomplete trailing escape (%) pattern
> Expected behavior:
> provided prefix must be listed
> Problem location:
> `AWSRequestAuthorizeSignatureV4.signForAuthorizationHeader()`
>  
> Multimap queryMap = 
> queryParser().apply(request.getEndpoint().getQuery());
> request.getEndpoint().getQuery() - returns a decoded query string
> however queryParser() require encoded one
> Fix:
> Multimap queryMap = 
> queryParser().apply(request.getEndpoint().getRawQuery());
> When jclouds generates a request to the AWS it encodes prefix (encoding all 
> special chars in it, including % and &), however calling `getQuery()` returns 
> decoded version of query string which lead to unpredictable behavior. 
> P.S. required patch in 2.5.x if possible
> PR https://github.com/apache/jclouds/pull/200



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Re: [PR] more validation for containerName and blobKey to avoid access escape [jclouds]

2024-04-14 Thread via GitHub 


gaul commented on code in PR #203:
URL: https://github.com/apache/jclouds/pull/203#discussion_r1564561709


##
apis/filesystem/src/main/java/org/jclouds/filesystem/predicates/validators/internal/FilesystemBlobKeyValidatorImpl.java:
##
@@ -38,6 +38,8 @@ public void validate(String name) throws 
IllegalArgumentException {
 //blobkey cannot start with / (or \ in Windows) character
 if (name.startsWith("\\") || name.startsWith("/"))
 throw new IllegalArgumentException("Blob key '" + name + "' cannot 
start with \\ or /");
+if (name.contains("../"))
+throw new IllegalArgumentException("Blob key '" + name + "' cannot 
contain ../");

Review Comment:
   Why is this different than the container name validator?  For robustness 
should this tokenize the path via `/` then check each component to see if one 
contains `.` or `..`?  This would allow keys like `..foo` to work.



##
apis/filesystem/src/main/java/org/jclouds/filesystem/strategy/internal/FilesystemStorageStrategyImpl.java:
##
@@ -691,6 +698,8 @@ public BlobAccess getBlobAccess(String containerName, 
String blobName) {
 
@Override
public void setBlobAccess(String container, String name, BlobAccess access) 
{
+  filesystemContainerNameValidator.validate(container);
+  filesystemBlobKeyValidator.validate(name);

Review Comment:
   Should this also apply to `getBlobAccess`, `putBlob`, and `removeBlob`?  
`getBlobKeysInsideContainer` too I guess.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: notifications-unsubscr...@jclouds.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org