Taras Bobrovytsky has posted comments on this change. ( http://gerrit.cloudera.org:8080/10275 )
Change subject: IMPALA-6949: Add the option to start the minicluster with EC enabled ...................................................................... Patch Set 2: (5 comments) Yes, nested data loading succeeds every time I tried it. http://gerrit.cloudera.org:8080/#/c/10275/2/bin/impala-config.sh File bin/impala-config.sh: http://gerrit.cloudera.org:8080/#/c/10275/2/bin/impala-config.sh@449 PS2, Line 449: elif [ "${TARGET_FILESYSTEM}" = "hdfs-ec" ]; then > By making this its own TARGET_FILESYSTEM, you end up disabling a bunch of t I don't think it makes sense to make erasure coding its own file system. I made a separate flag for it. http://gerrit.cloudera.org:8080/#/c/10275/2/bin/impala-config.sh@454 PS2, Line 454: echo "Valid values are: hdfs, isilon, s3, local" > Please update? No need to update any more. http://gerrit.cloudera.org:8080/#/c/10275/2/bin/run-all-tests.sh File bin/run-all-tests.sh: http://gerrit.cloudera.org:8080/#/c/10275/2/bin/run-all-tests.sh@71 PS2, Line 71: FE_TEST=false > Add a comment about why? Done http://gerrit.cloudera.org:8080/#/c/10275/2/testdata/bin/setup-hdfs-env.sh File testdata/bin/setup-hdfs-env.sh: http://gerrit.cloudera.org:8080/#/c/10275/2/testdata/bin/setup-hdfs-env.sh@80 PS2, Line 80: hdfs ec -unsetPolicy -path "${HDFS_EC_PATH:=/test-warehouse/}" > Do you know what this does underneath the covers? Do we need to block while This does not convert the existing files in the directory. It only affects the files that will be placed in the directory in the future. This is why it is possible to have some files erasure coded and some not in the same directory. http://gerrit.cloudera.org:8080/#/c/10275/2/testdata/cluster/node_templates/common/etc/hadoop/conf/hdfs-site.xml.tmpl File testdata/cluster/node_templates/common/etc/hadoop/conf/hdfs-site.xml.tmpl: http://gerrit.cloudera.org:8080/#/c/10275/2/testdata/cluster/node_templates/common/etc/hadoop/conf/hdfs-site.xml.tmpl@26 PS2, Line 26: <name>cloudera.erasure_coding.enabled</name> > What's this for? This is needed because running the following command hdfs ec -enablePolicy -policy RS-3-2-1024k Causes this error: RemoteException: enableErasureCodingPolicy is not allowed because cloudera.erasure_coding.enabled=false -- To view, visit http://gerrit.cloudera.org:8080/10275 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I397aed491354be21b0a8441ca671232dca25146c Gerrit-Change-Number: 10275 Gerrit-PatchSet: 2 Gerrit-Owner: Taras Bobrovytsky <tbobrovyt...@cloudera.com> Gerrit-Reviewer: Philip Zeyliger <phi...@cloudera.com> Gerrit-Reviewer: Taras Bobrovytsky <tbobrovyt...@cloudera.com> Gerrit-Reviewer: Tianyi Wang <tw...@cloudera.com> Gerrit-Comment-Date: Wed, 02 May 2018 21:23:44 +0000 Gerrit-HasComments: Yes
