date:20160923

Re: Zip Exception since commit da4098b9

2016-09-23 Thread Madhusudan Ramanna

Hitesh and Zhiyuan,
Apache snapshots doesn't seem to have tez-dist 
http://repository.apache.org/content/groups/snapshots/org/apache/tez/tez-dist/

The last one seems to be 0.2.0-SNAPSHOT
Should I just download based on the commit and recompile ? 
thanks,Madhu

On Friday, September 23, 2016 5:19 PM, Hitesh Shah  
wrote:
 

 Hello Madhusudan,

If you look at the MANIFEST.MF inside any of the tez jars, it will provide the 
commit hash via the SCM-Revision field.

The tez client and the DAGAppMaster also log this info at runtime.

— Hitesh 

> On Sep 23, 2016, at 4:08 PM, Madhusudan Ramanna  wrote:
> 
> Zhiyuan,
> 
> We just pulled down the latest snapshot from Apache repository.  Question, is 
> how can I figure out branch and commit information from the snapshot artifact 
> ?
> 
> thanks,
> Madhu
> 
> 
> On Friday, September 23, 2016 10:38 AM, zhiyuan yang  
> wrote:
> 
> 
> Hi Madhu,
> 
> It looks like a Inflater-Deflater mismatch to me. From stack traces I see you 
> cherry-picked this patch instead of using master branch.
> Would you mind double check whether the patch is correctly cherry-picked?
> 
> Thanks!
> Zhiyuan
> 
>> On Sep 23, 2016, at 10:21 AM, Madhusudan Ramanna  wrote:
>> 
>> Hello,
>> 
>> We're using the Apache snapshot repository to pull latest tez snapshots. 
>> 
>> We've started seeing this exception:
>> 
>> org.apache.tez.dag.api.TezUncheckedException: java.util.zip.ZipException: 
>> incorrect header check
>> at 
>> org.apache.tez.dag.library.vertexmanager.ShuffleVertexManager.handleVertexManagerEvent(ShuffleVertexManager.java:622)
>> at 
>> org.apache.tez.dag.library.vertexmanager.ShuffleVertexManager.onVertexManagerEventReceived(ShuffleVertexManager.java:579)
>> at 
>> org.apache.tez.dag.app.dag.impl.VertexManager$VertexManagerEventReceived.invoke(VertexManager.java:606)
>> at 
>> org.apache.tez.dag.app.dag.impl.VertexManager$VertexManagerEvent$1.run(VertexManager.java:647)
>> at 
>> org.apache.tez.dag.app.dag.impl.VertexManager$VertexManagerEvent$1.run(VertexManager.java:642)
>> at java.security.AccessController.doPrivileged(Native Method)
>> at javax.security.auth.Subject.doAs(Subject.java:422)
>> at 
>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
>> at 
>> org.apache.tez.dag.app.dag.impl.VertexManager$VertexManagerEvent.call(VertexManager.java:642)
>> at 
>> org.apache.tez.dag.app.dag.impl.VertexManager$VertexManagerEvent.call(VertexManager.java:631)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> at 
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> at 
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> at java.lang.Thread.run(Thread.java:745)
>> Caused by: java.util.zip.ZipException: incorrect header check
>> at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:164)
>> at java.io.FilterInputStream.read(FilterInputStream.java:107)
>> at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1792)
>> at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1769)
>> at org.apache.commons.io.IOUtils.copy(IOUtils.java:1744)
>> at org.apache.commons.io.IOUtils.toByteArray(IOUtils.java:462)
>> 
>> 
>> since this commit
>> 
>> https://github.com/apache/tez/commit/da4098b9d6f72e6d4aacc1623622a0875408d2ba
>> 
>> 
>> Wanted to bring this to your attention. For now we've locked the snapshot 
>> version down.
>> 
>> thanks,
>> Madhu
> 
> 
>

Re: Zip Exception since commit da4098b9

2016-09-23 Thread Hitesh Shah

Hello Madhusudan,

If you look at the MANIFEST.MF inside any of the tez jars, it will provide the 
commit hash via the SCM-Revision field.

The tez client and the DAGAppMaster also log this info at runtime.

— Hitesh 

> On Sep 23, 2016, at 4:08 PM, Madhusudan Ramanna  wrote:
> 
> Zhiyuan,
> 
> We just pulled down the latest snapshot from Apache repository.   Question, 
> is how can I figure out branch and commit information from the snapshot 
> artifact ?
> 
> thanks,
> Madhu
> 
> 
> On Friday, September 23, 2016 10:38 AM, zhiyuan yang  
> wrote:
> 
> 
> Hi Madhu,
> 
> It looks like a Inflater-Deflater mismatch to me. From stack traces I see you 
> cherry-picked this patch instead of using master branch.
> Would you mind double check whether the patch is correctly cherry-picked?
> 
> Thanks!
> Zhiyuan
> 
>> On Sep 23, 2016, at 10:21 AM, Madhusudan Ramanna  wrote:
>> 
>> Hello,
>> 
>> We're using the Apache snapshot repository to pull latest tez snapshots. 
>> 
>> We've started seeing this exception:
>> 
>> org.apache.tez.dag.api.TezUncheckedException: java.util.zip.ZipException: 
>> incorrect header check
>> at 
>> org.apache.tez.dag.library.vertexmanager.ShuffleVertexManager.handleVertexManagerEvent(ShuffleVertexManager.java:622)
>> at 
>> org.apache.tez.dag.library.vertexmanager.ShuffleVertexManager.onVertexManagerEventReceived(ShuffleVertexManager.java:579)
>> at 
>> org.apache.tez.dag.app.dag.impl.VertexManager$VertexManagerEventReceived.invoke(VertexManager.java:606)
>> at 
>> org.apache.tez.dag.app.dag.impl.VertexManager$VertexManagerEvent$1.run(VertexManager.java:647)
>> at 
>> org.apache.tez.dag.app.dag.impl.VertexManager$VertexManagerEvent$1.run(VertexManager.java:642)
>> at java.security.AccessController.doPrivileged(Native Method)
>> at javax.security.auth.Subject.doAs(Subject.java:422)
>> at 
>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
>> at 
>> org.apache.tez.dag.app.dag.impl.VertexManager$VertexManagerEvent.call(VertexManager.java:642)
>> at 
>> org.apache.tez.dag.app.dag.impl.VertexManager$VertexManagerEvent.call(VertexManager.java:631)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> at 
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> at 
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> at java.lang.Thread.run(Thread.java:745)
>> Caused by: java.util.zip.ZipException: incorrect header check
>> at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:164)
>> at java.io.FilterInputStream.read(FilterInputStream.java:107)
>> at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1792)
>> at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1769)
>> at org.apache.commons.io.IOUtils.copy(IOUtils.java:1744)
>> at org.apache.commons.io.IOUtils.toByteArray(IOUtils.java:462)
>> 
>> 
>> since this commit
>> 
>> https://github.com/apache/tez/commit/da4098b9d6f72e6d4aacc1623622a0875408d2ba
>> 
>> 
>> Wanted to bring this to your attention. For now we've locked the snapshot 
>> version down.
>> 
>> thanks,
>> Madhu
> 
> 
>

Re: Zip Exception since commit da4098b9

2016-09-23 Thread Madhusudan Ramanna

Zhiyuan,
We just pulled down the latest snapshot from Apache repository.   Question, is 
how can I figure out branch and commit information from the snapshot artifact ?
thanks,Madhu 

On Friday, September 23, 2016 10:38 AM, zhiyuan yang  
wrote:
 

 Hi Madhu,
It looks like a Inflater-Deflater mismatch to me. From stack traces I see you 
cherry-picked this patch instead of using master branch.Would you mind double 
check whether the patch is correctly cherry-picked?
Thanks!Zhiyuan

On Sep 23, 2016, at 10:21 AM, Madhusudan Ramanna  wrote:
Hello,
We're using the Apache snapshot repository to pull latest tez snapshots. 
We've started seeing this exception:
org.apache.tez.dag.api.TezUncheckedException: java.util.zip.ZipException: 
incorrect header check
at 
org.apache.tez.dag.library.vertexmanager.ShuffleVertexManager.handleVertexManagerEvent(ShuffleVertexManager.java:622)
at 
org.apache.tez.dag.library.vertexmanager.ShuffleVertexManager.onVertexManagerEventReceived(ShuffleVertexManager.java:579)
at 
org.apache.tez.dag.app.dag.impl.VertexManager$VertexManagerEventReceived.invoke(VertexManager.java:606)
at 
org.apache.tez.dag.app.dag.impl.VertexManager$VertexManagerEvent$1.run(VertexManager.java:647)
at 
org.apache.tez.dag.app.dag.impl.VertexManager$VertexManagerEvent$1.run(VertexManager.java:642)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at 
org.apache.tez.dag.app.dag.impl.VertexManager$VertexManagerEvent.call(VertexManager.java:642)
at 
org.apache.tez.dag.app.dag.impl.VertexManager$VertexManagerEvent.call(VertexManager.java:631)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.util.zip.ZipException: incorrect header check
at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:164)
at java.io.FilterInputStream.read(FilterInputStream.java:107)
at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1792)
at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1769)
at org.apache.commons.io.IOUtils.copy(IOUtils.java:1744)
at org.apache.commons.io.IOUtils.toByteArray(IOUtils.java:462)


since this commit
https://github.com/apache/tez/commit/da4098b9d6f72e6d4aacc1623622a0875408d2ba


Wanted to bring this to your attention. For now we've locked the snapshot 
version down.
thanks,Madhu

Re: Zip Exception since commit da4098b9

2016-09-23 Thread Hitesh Shah

Hello Madhusudan 

Thanks for reporting the issue. Would you mind filing a bug at 
https://issues.apache.org/jira/browse/tez with the application logs and tez 
configs attached? If you have a simple dag/job example that reproduces the 
behavior that would be great too.

thanks
— Hitesh

> On Sep 23, 2016, at 10:38 AM, zhiyuan yang  wrote:
> 
> Hi Madhu,
> 
> It looks like a Inflater-Deflater mismatch to me. From stack traces I see you 
> cherry-picked this patch instead of using master branch.
> Would you mind double check whether the patch is correctly cherry-picked?
> 
> Thanks!
> Zhiyuan
> 
>> On Sep 23, 2016, at 10:21 AM, Madhusudan Ramanna  wrote:
>> 
>> Hello,
>> 
>> We're using the Apache snapshot repository to pull latest tez snapshots. 
>> 
>> We've started seeing this exception:
>> 
>> org.apache.tez.dag.api.TezUncheckedException: java.util.zip.ZipException: 
>> incorrect header check
>> at 
>> org.apache.tez.dag.library.vertexmanager.ShuffleVertexManager.handleVertexManagerEvent(ShuffleVertexManager.java:622)
>> at 
>> org.apache.tez.dag.library.vertexmanager.ShuffleVertexManager.onVertexManagerEventReceived(ShuffleVertexManager.java:579)
>> at 
>> org.apache.tez.dag.app.dag.impl.VertexManager$VertexManagerEventReceived.invoke(VertexManager.java:606)
>> at 
>> org.apache.tez.dag.app.dag.impl.VertexManager$VertexManagerEvent$1.run(VertexManager.java:647)
>> at 
>> org.apache.tez.dag.app.dag.impl.VertexManager$VertexManagerEvent$1.run(VertexManager.java:642)
>> at java.security.AccessController.doPrivileged(Native Method)
>> at javax.security.auth.Subject.doAs(Subject.java:422)
>> at 
>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
>> at 
>> org.apache.tez.dag.app.dag.impl.VertexManager$VertexManagerEvent.call(VertexManager.java:642)
>> at 
>> org.apache.tez.dag.app.dag.impl.VertexManager$VertexManagerEvent.call(VertexManager.java:631)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>> at 
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> at 
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> at java.lang.Thread.run(Thread.java:745)
>> Caused by: java.util.zip.ZipException: incorrect header check
>> at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:164)
>> at java.io.FilterInputStream.read(FilterInputStream.java:107)
>> at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1792)
>> at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1769)
>> at org.apache.commons.io.IOUtils.copy(IOUtils.java:1744)
>> at org.apache.commons.io.IOUtils.toByteArray(IOUtils.java:462)
>> 
>> 
>> since this commit
>> 
>> https://github.com/apache/tez/commit/da4098b9d6f72e6d4aacc1623622a0875408d2ba
>> 
>> 
>> Wanted to bring this to your attention. For now we've locked the snapshot 
>> version down.
>> 
>> thanks,
>> Madhu
>

Zip Exception since commit da4098b9

2016-09-23 Thread Madhusudan Ramanna

Hello,
We're using the Apache snapshot repository to pull latest tez snapshots. 
We've started seeing this exception:
org.apache.tez.dag.api.TezUncheckedException: java.util.zip.ZipException: 
incorrect header check
at 
org.apache.tez.dag.library.vertexmanager.ShuffleVertexManager.handleVertexManagerEvent(ShuffleVertexManager.java:622)
at 
org.apache.tez.dag.library.vertexmanager.ShuffleVertexManager.onVertexManagerEventReceived(ShuffleVertexManager.java:579)
at 
org.apache.tez.dag.app.dag.impl.VertexManager$VertexManagerEventReceived.invoke(VertexManager.java:606)
at 
org.apache.tez.dag.app.dag.impl.VertexManager$VertexManagerEvent$1.run(VertexManager.java:647)
at 
org.apache.tez.dag.app.dag.impl.VertexManager$VertexManagerEvent$1.run(VertexManager.java:642)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at 
org.apache.tez.dag.app.dag.impl.VertexManager$VertexManagerEvent.call(VertexManager.java:642)
at 
org.apache.tez.dag.app.dag.impl.VertexManager$VertexManagerEvent.call(VertexManager.java:631)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.util.zip.ZipException: incorrect header check
at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:164)
at java.io.FilterInputStream.read(FilterInputStream.java:107)
at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1792)
at org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1769)
at org.apache.commons.io.IOUtils.copy(IOUtils.java:1744)
at org.apache.commons.io.IOUtils.toByteArray(IOUtils.java:462)


since this commit
https://github.com/apache/tez/commit/da4098b9d6f72e6d4aacc1623622a0875408d2ba


Wanted to bring this to your attention. For now we've locked the snapshot 
version down.
thanks,Madhu

Re: Zip Exception since commit da4098b9

Re: Zip Exception since commit da4098b9

Re: Zip Exception since commit da4098b9

Re: Zip Exception since commit da4098b9

Zip Exception since commit da4098b9

5 matches

Site Navigation

Mail list logo

Footer information