[
https://issues.apache.org/jira/browse/BLUR-441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14720413#comment-14720413
]
Tom Hood edited comment on BLUR-441 at 8/28/15 7:05 PM:
--------------------------------------------------------
I tried your random UUID version of the test modified slightly to use the blur
cluster running on my laptop (rather than MiniCluster) and it fails right away
with the same kind of corruption.
Have you already tried the original code I uploaded with 0.2.4 or were you
always using the newest blur code?
I created the empty table from the blur shell as before with 1 shard and not
specifying any schema.
Laptop Environment
- Dell XPS L702X, 8GB ram, i7-2670QM CPU @ 2.2GHZ (4 cores, 8 threads), Windows
7 SP1
- CentOS 6.4 vm running with Oracle VirtualBox. VM is allocated 6 processors
and 4GB ram
- hadoop version 0.20.205.0
- zookeeper version 3.4.5
- blur 0.2.4 (server setup to run with java 1.8.0_45; client/test setup to run
with the same, but also tried 1.6.0_45)
Work Environment
- blur 0.2.4 (server setup to run with some flavor of java 1.7; client/test
setup to run with the same)
- CDH 5.1.0 hadoop stack
- RedHat 6.3 Santiago
- (since going from memory, can't remember anything else about it)
I tried building the blur code on my laptop with the tests enabled, but I had
various issues doing that. I started with a git clone and was messing around
with the build_release.sh script (had to edit the script due to the version
check it does since I am using mvn 3.3.3 which wants java 1.7 or later and the
build_release.sh check is forcing the version to 1.6 or it fails, although the
commentary in the build_release.sh suggests that 1.6 or later should be
allowed). So I commented out the version check and attempted to build with
java 1.8.0_45 (I don't have 1.7 on my laptop right now). I also changed the
hadoop1.versions to 0.20.205.0 and the release_profiles to just hadoop1. I
tried that, but got a compile error:
[ERROR]
/srv/local/blur/incubator-blur/blur-kvs/src/main/java/org/apache/blur/kvs/HdfsKeyValueStore.java:[44,29]
package org.apache.commons.io does not exist
[ERROR]
/srv/local/blur/incubator-blur/blur-kvs/src/main/java/org/apache/blur/kvs/HdfsKeyValueStore.java:[502,13]
cannot find symbol
symbol: variable IOUtils
location: class org.apache.blur.kvs.HdfsKeyValueStore
Rather than look into this, however, I then realized your "Getting Started"
webpage just suggests doing a "mvn install -DskipTests -Dhadoop1" rather than
using the build_release.sh. So I tried that without the -DskipTests, because
I'd like to see if the random UUID test you added works on my laptop as part of
the build. However, didn't get that far, because of a
org.apache.blur.kvs.HdfsKeyValueStoreTest test failure using a hadoop mini
cluster. Error looks like:
15/08/28 10:29:07 INFO kvs.HdfsKeyValueStore: Opening for writing
[hdfs://localhost.localdomain:58684/test/000000000001].
15/08/28 10:29:07 INFO hdfs.StateChange: BLOCK* allocateBlock:
/test/000000000001. blk_-7602275106474658451_1001
15/08/28 10:29:07 INFO hdfs.DFSClient: Exception in createBlockOutputStream
<mylaptopIP>:53612 java.net.ConnectException: Connection refused
15/08/28 10:29:07 INFO hdfs.DFSClient: Abandoning blk_-7602275106474658451_1001
15/08/28 10:29:07 INFO hdfs.DFSClient: Excluding datanode <mylaptopIP>:53612
15/08/28 10:29:07 WARN namenode.FSNamesystem: Not able to place enough
replicas, still in need of 1 to reach 1
Not able to place enough replicas
15/08/28 10:29:07 ERROR security.UserGroupInformation:
PriviledgedActionException as:hood cause:java.io.IOException: File
/test/000000000001 could only be replicated to 0 nodes, instead of 1
No time to track this down right now. However, I added back the -DskipTests
and it built okay. Now that I have the latest blur jars built, I can later try
running from eclipse your random UUID test with MiniCluster.
was (Author: tom hood):
I tried your random UUID version of the test modified slightly to use the blur
cluster running on my laptop (rather than MiniCluster) and it fails right away
with the same kind of corruption.
Have you already tried the original code I uploaded with 0.2.4 or were you
always using the newest blur code?
The blur "cluster" on my laptop is just 1 shard server and 1 controller. I
created the empty table from the blur shell as before with 1 shard and not
specifying any schema.
java version didn't seem to matter: tried versions of 1.6 and 1.8 for the test
program. Only tried the 1.8 for JAVA_HOME running of the shard+controller
servers. However, the test failure on my dev machine at work ran with a 1.7
version (both client and server).
Laptop Environment
- Dell XPS L702X, 8GB ram, i7-2670QM CPU @ 2.2GHZ (4 cores, 8 threads), Windows
7 SP1
- CentOS 6.4 vm running with Oracle VirtualBox. VM is allocated 6 processors
and 4GB ram
- hadoop version 0.20.205.0
- zookeeper version 3.4.5
- blur 0.2.4 (server setup to run with java 1.8.0_45; client/test setup to run
with the same, but also tried 1.6.0_45)
Work Environment
- blur 0.2.4 (server setup to run with some flavor of java 1.7; client/test
setup to run with the same)
- CDH 5.1.0 hadoop stack
- RedHat 6.3 Santiago
- (since going from memory, can't remember anything else about it)
I tried building the blur code on my laptop with the tests enabled, but I had
various issues doing that. I started with a git clone and was messing around
with the build_release.sh script (had to edit the script due to the version
check it does since I am using mvn 3.3.3 which wants java 1.7 or later and the
build_release.sh check is forcing the version to 1.6 or it fails, although the
commentary in the build_release.sh suggests that 1.6 or later should be
allowed). So I commented out the version check and attempted to build with
java 1.8.0_45 (I don't have 1.7 on my laptop right now). I also changed the
hadoop1.versions to 0.20.205.0 and the release_profiles to just hadoop1. I
tried that, but got a compile error:
[ERROR]
/srv/local/blur/incubator-blur/blur-kvs/src/main/java/org/apache/blur/kvs/HdfsKeyValueStore.java:[44,29]
package org.apache.commons.io does not exist
[ERROR]
/srv/local/blur/incubator-blur/blur-kvs/src/main/java/org/apache/blur/kvs/HdfsKeyValueStore.java:[502,13]
cannot find symbol
symbol: variable IOUtils
location: class org.apache.blur.kvs.HdfsKeyValueStore
Rather than look into this, however, I then realized your "Getting Started"
webpage just suggests doing a "mvn install -DskipTests -Dhadoop1" rather than
using the build_release.sh. So I tried that without the -DskipTests, because
I'd like to see if the random UUID test you added works on my laptop as part of
the build. However, didn't get that far, because of a
org.apache.blur.kvs.HdfsKeyValueStoreTest test failure using a hadoop mini
cluster. Error looks like:
15/08/28 10:29:07 INFO kvs.HdfsKeyValueStore: Opening for writing
[hdfs://localhost.localdomain:58684/test/000000000001].
15/08/28 10:29:07 INFO hdfs.StateChange: BLOCK* allocateBlock:
/test/000000000001. blk_-7602275106474658451_1001
15/08/28 10:29:07 INFO hdfs.DFSClient: Exception in createBlockOutputStream
<mylaptopIP>:53612 java.net.ConnectException: Connection refused
15/08/28 10:29:07 INFO hdfs.DFSClient: Abandoning blk_-7602275106474658451_1001
15/08/28 10:29:07 INFO hdfs.DFSClient: Excluding datanode <mylaptopIP>:53612
15/08/28 10:29:07 WARN namenode.FSNamesystem: Not able to place enough
replicas, still in need of 1 to reach 1
Not able to place enough replicas
15/08/28 10:29:07 ERROR security.UserGroupInformation:
PriviledgedActionException as:hood cause:java.io.IOException: File
/test/000000000001 could only be replicated to 0 nodes, instead of 1
No time to track this down right now. However, I added back the -DskipTests
and it built okay. Now that I have the latest blur jars built, I can later try
running from eclipse your random UUID test with MiniCluster.
> Row Mutation Bug
> ----------------
>
> Key: BLUR-441
> URL: https://issues.apache.org/jira/browse/BLUR-441
> Project: Apache Blur
> Issue Type: Bug
> Components: Blur
> Affects Versions: 0.2.4
> Reporter: Aaron McCurry
> Assignee: Aaron McCurry
> Priority: Blocker
> Fix For: 0.2.4
>
> Attachments: BlurMutationBug.java
>
>
> Reported on blur-user mail list:
> http://mail-archives.apache.org/mod_mbox/incubator-blur-user/201508.mbox/<CABO0Qi6m6xR60R64XBpoq7T0Q4GxxAJDcetV_-6PAmg32FdNxg%40mail.gmail.com>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
