[GitHub] carbondata pull request #1239: [CARBONDATA-1343] Hive can't query data when ...
Github user cenyuhai closed the pull request at: https://github.com/apache/carbondata/pull/1239 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] carbondata pull request #1239: [CARBONDATA-1343] Hive can't query data when ...
Github user cenyuhai commented on a diff in the pull request: https://github.com/apache/carbondata/pull/1239#discussion_r132695250 --- Diff: integration/hive/src/main/java/org/apache/carbondata/hive/MapredCarbonInputFormat.java --- @@ -84,47 +92,66 @@ * @param configuration * @throws IOException */ - private static void populateCarbonTable(Configuration configuration, String paths) + private CarbonTable populateCarbonTable(Configuration configuration) throws IOException { -String dirs = configuration.get(INPUT_DIR, ""); -String[] inputPaths = StringUtils.split(dirs); -String validInputPath = null; +TableInfo tableInfo = getTableInfo(configuration); +CarbonTable carbonTable = null; +if (tableInfo != null) { + carbonTable = CarbonTable.buildFromTableInfo(tableInfo); + CarbonMetadata.getInstance().addCarbonTable(carbonTable); + return carbonTable; +} +String inputDir = configuration.get(INPUT_DIR, ""); +String[] inputPaths = StringUtils.split(inputDir); if (inputPaths.length == 0) { throw new InvalidPathException("No input paths specified in job"); -} else { - if (paths != null) { -for (String inputPath : inputPaths) { - if (paths.startsWith(inputPath)) { -validInputPath = inputPath; -break; - } -} - } } +Arrays.sort(inputPaths); +String tablePath = inputPaths[0].replace("file:", ""); --- End diff -- because of another issue, my pr is conflict with https://github.com/apache/carbondata/pull/1231 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] carbondata pull request #1239: [CARBONDATA-1343] Hive can't query data when ...
Github user cenyuhai commented on a diff in the pull request: https://github.com/apache/carbondata/pull/1239#discussion_r132691664 --- Diff: integration/hive/src/main/java/org/apache/carbondata/hive/MapredCarbonInputFormat.java --- @@ -84,47 +92,66 @@ * @param configuration * @throws IOException */ - private static void populateCarbonTable(Configuration configuration, String paths) + private CarbonTable populateCarbonTable(Configuration configuration) throws IOException { -String dirs = configuration.get(INPUT_DIR, ""); -String[] inputPaths = StringUtils.split(dirs); -String validInputPath = null; +TableInfo tableInfo = getTableInfo(configuration); +CarbonTable carbonTable = null; +if (tableInfo != null) { + carbonTable = CarbonTable.buildFromTableInfo(tableInfo); + CarbonMetadata.getInstance().addCarbonTable(carbonTable); + return carbonTable; +} +String inputDir = configuration.get(INPUT_DIR, ""); +String[] inputPaths = StringUtils.split(inputDir); if (inputPaths.length == 0) { throw new InvalidPathException("No input paths specified in job"); -} else { - if (paths != null) { -for (String inputPath : inputPaths) { - if (paths.startsWith(inputPath)) { -validInputPath = inputPath; -break; - } -} - } } +Arrays.sort(inputPaths); --- End diff -- because there are serveral paths, the sortest path are the table path, so I sort all and then get the first one --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] carbondata pull request #1239: [CARBONDATA-1343] Hive can't query data when ...
Github user xuchuanyin commented on a diff in the pull request: https://github.com/apache/carbondata/pull/1239#discussion_r132685712 --- Diff: integration/hive/src/main/java/org/apache/carbondata/hive/MapredCarbonInputFormat.java --- @@ -84,47 +92,66 @@ * @param configuration * @throws IOException */ - private static void populateCarbonTable(Configuration configuration, String paths) + private CarbonTable populateCarbonTable(Configuration configuration) throws IOException { -String dirs = configuration.get(INPUT_DIR, ""); -String[] inputPaths = StringUtils.split(dirs); -String validInputPath = null; +TableInfo tableInfo = getTableInfo(configuration); +CarbonTable carbonTable = null; +if (tableInfo != null) { + carbonTable = CarbonTable.buildFromTableInfo(tableInfo); + CarbonMetadata.getInstance().addCarbonTable(carbonTable); + return carbonTable; +} +String inputDir = configuration.get(INPUT_DIR, ""); +String[] inputPaths = StringUtils.split(inputDir); if (inputPaths.length == 0) { throw new InvalidPathException("No input paths specified in job"); -} else { - if (paths != null) { -for (String inputPath : inputPaths) { - if (paths.startsWith(inputPath)) { -validInputPath = inputPath; -break; - } -} - } } +Arrays.sort(inputPaths); --- End diff -- why a sort is needed here? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] carbondata pull request #1239: [CARBONDATA-1343] Hive can't query data when ...
Github user xuchuanyin commented on a diff in the pull request: https://github.com/apache/carbondata/pull/1239#discussion_r132685377 --- Diff: integration/hive/src/main/java/org/apache/carbondata/hive/MapredCarbonInputFormat.java --- @@ -84,47 +92,66 @@ * @param configuration * @throws IOException */ - private static void populateCarbonTable(Configuration configuration, String paths) + private CarbonTable populateCarbonTable(Configuration configuration) throws IOException { -String dirs = configuration.get(INPUT_DIR, ""); -String[] inputPaths = StringUtils.split(dirs); -String validInputPath = null; +TableInfo tableInfo = getTableInfo(configuration); +CarbonTable carbonTable = null; +if (tableInfo != null) { + carbonTable = CarbonTable.buildFromTableInfo(tableInfo); + CarbonMetadata.getInstance().addCarbonTable(carbonTable); + return carbonTable; +} +String inputDir = configuration.get(INPUT_DIR, ""); +String[] inputPaths = StringUtils.split(inputDir); if (inputPaths.length == 0) { throw new InvalidPathException("No input paths specified in job"); -} else { - if (paths != null) { -for (String inputPath : inputPaths) { - if (paths.startsWith(inputPath)) { -validInputPath = inputPath; -break; - } -} - } } +Arrays.sort(inputPaths); +String tablePath = inputPaths[0].replace("file:", ""); --- End diff -- Is this a mistake? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---