Re: Impala Toolchain Build: Unsupported options error
Hi Todd, Surely, I'm working to get kudu ported on to ppc64le. As you suggested, I'll start the discussion thread on [email protected] Regards, Valencia From: Todd Lipcon To: "dev@impala" Cc: Manish Patil/Austin/Contr/IBM@IBMUS, Sudarshan Jagadale/Austin/Contr/IBM@IBMUS, Nishidha Panpaliya/Austin/Contr/IBM@IBMUS, Valencia Serrao/Austin/Contr/IBM@IBMUS Date: 09/23/2016 01:07 AM Subject:Re: Impala Toolchain Build: Unsupported options error Jumping in from the Kudu side: we've made no effort so far to get Kudu running on ppc64le, and we probably have a decent amount of work to get it stable. Given you're focusing on little-endian, it probably wouldn't be awful, but we might be silently relying on x86's total store order in some spots, whereas PPC's memory model is significantly weaker. If you're interested in contributing to such a port, feel free to reach out to [email protected] with any questions. -Todd On Thu, Sep 22, 2016 at 9:16 AM, Matthew Jacobs wrote: I should add that the last option about "disabling" kudu we've found to be painful in practice and may cause you to have some issues running/testing. If it's possible to move forward with the toolchain compilation I would strongly recommend that. On Thu, Sep 22, 2016 at 9:13 AM, Matthew Jacobs wrote: > Any modifications you make to the Kudu build would probably need to be > checked by the folks working on the Kudu project. Impala builds Kudu > for 2 reasons: 1) for the client which it links against (build time > dependency) 2) for the dev environment (dev and test dependency). > > What are you trying to do? If you want Impala to work on ppc64le, then > you'll probably eventually want Kudu to work on ppc64le as well, and > that might take a lot more work/testing. > > If you want the impala that you build to be able to communicate with > Kudu at all (whether or not Kudu runs on the same platform), you do > need to compile in the client as you're currently doing. In that case, > you should ask the Kudu authors at [email protected]. > > If you're OK with not supporting any Kudu for now, then you can > "disable" Kudu support in the build/runtime. To do this, you should: > 1) in native-toolchain: > get source/kudu/build.sh is_supported_platform() to return false > for ppc64le > that results in a stub being built instead of all of kudu > 2) in Impala: changing impala-config.sh so IS_KUDU_SUPPORTED is false > for ppc64le. > > On Wed, Sep 21, 2016 at 10:45 PM, Valencia Serrao wrote: >> >> Hi All, >> >> I am building Impala-toolchain on ppc64le. However, while building Kudu's >> thirdparty "libstdc++-v3", I have encountered following errors: >> Error1: clang-3.8: error: unsupported option '-print-multi-os-directory' >> Error2: clang-3.8: error: unknown argument: '-mlong-double-64' >> >> I couldn't find any documentation to find the equivalent options to be used >> on ppc64le. To get around the Error1, , I have removed the >> -print-multi-os-directory option and continued with the build. >> >> The build has surely progressed with this work-around, however, I need to >> know: >> 1. What is the impact of removing "-print-multi-os-directory" ? Will it >> significantly affect the functionality for Impala ? >> 2. Can I take the same approach for the Error2 ? What will be the impact on >> functionality ? >> 3. If removing the above two options has a significant impact, then, could >> you share the documentation/equivalent options to be used for ppc64le ? >> >> Any pointers on this issue will be helpful. >> >> Regards, >> Valencia -- Todd Lipcon Software Engineer, Cloudera
Re: Impala Toolchain Build: Unsupported options error
Hi Matthew, Thanks for the insight and the work-around you listed. With regard kudu stub, our colleague Nishidha, had tried building it. But we had to face many issues with that approach. However, we need Impala to work full-fledged on ppc64le. For that to happen, as you also mentioned, real kudu (not kudu stub) also should work on ppc64le. Surely, I will continue on the toolchain compilation. Thanks once again for the quick response. Regards, Valencia From: Matthew Jacobs To: [email protected] Cc: Manish Patil/Austin/Contr/IBM@IBMUS, Sudarshan Jagadale/Austin/Contr/IBM@IBMUS, Nishidha Panpaliya/Austin/Contr/IBM@IBMUS, Valencia Serrao/Austin/Contr/IBM@IBMUS Date: 09/22/2016 09:46 PM Subject:Re: Impala Toolchain Build: Unsupported options error I should add that the last option about "disabling" kudu we've found to be painful in practice and may cause you to have some issues running/testing. If it's possible to move forward with the toolchain compilation I would strongly recommend that. On Thu, Sep 22, 2016 at 9:13 AM, Matthew Jacobs wrote: > Any modifications you make to the Kudu build would probably need to be > checked by the folks working on the Kudu project. Impala builds Kudu > for 2 reasons: 1) for the client which it links against (build time > dependency) 2) for the dev environment (dev and test dependency). > > What are you trying to do? If you want Impala to work on ppc64le, then > you'll probably eventually want Kudu to work on ppc64le as well, and > that might take a lot more work/testing. > > If you want the impala that you build to be able to communicate with > Kudu at all (whether or not Kudu runs on the same platform), you do > need to compile in the client as you're currently doing. In that case, > you should ask the Kudu authors at [email protected]. > > If you're OK with not supporting any Kudu for now, then you can > "disable" Kudu support in the build/runtime. To do this, you should: > 1) in native-toolchain: > get source/kudu/build.sh is_supported_platform() to return false > for ppc64le > that results in a stub being built instead of all of kudu > 2) in Impala: changing impala-config.sh so IS_KUDU_SUPPORTED is false > for ppc64le. > > On Wed, Sep 21, 2016 at 10:45 PM, Valencia Serrao wrote: >> >> Hi All, >> >> I am building Impala-toolchain on ppc64le. However, while building Kudu's >> thirdparty "libstdc++-v3", I have encountered following errors: >> Error1: clang-3.8: error: unsupported option '-print-multi-os-directory' >> Error2: clang-3.8: error: unknown argument: '-mlong-double-64' >> >> I couldn't find any documentation to find the equivalent options to be used >> on ppc64le. To get around the Error1, , I have removed the >> -print-multi-os-directory option and continued with the build. >> >> The build has surely progressed with this work-around, however, I need to >> know: >> 1. What is the impact of removing "-print-multi-os-directory" ? Will it >> significantly affect the functionality for Impala ? >> 2. Can I take the same approach for the Error2 ? What will be the impact on >> functionality ? >> 3. If removing the above two options has a significant impact, then, could >> you share the documentation/equivalent options to be used for ppc64le ? >> >> Any pointers on this issue will be helpful. >> >> Regards, >> Valencia
Re: Impala Toolchain Build: Unsupported options error
Jumping in from the Kudu side: we've made no effort so far to get Kudu running on ppc64le, and we probably have a decent amount of work to get it stable. Given you're focusing on little-endian, it probably wouldn't be awful, but we might be silently relying on x86's total store order in some spots, whereas PPC's memory model is significantly weaker. If you're interested in contributing to such a port, feel free to reach out to [email protected] with any questions. -Todd On Thu, Sep 22, 2016 at 9:16 AM, Matthew Jacobs wrote: > I should add that the last option about "disabling" kudu we've found > to be painful in practice and may cause you to have some issues > running/testing. If it's possible to move forward with the toolchain > compilation I would strongly recommend that. > > On Thu, Sep 22, 2016 at 9:13 AM, Matthew Jacobs wrote: > > Any modifications you make to the Kudu build would probably need to be > > checked by the folks working on the Kudu project. Impala builds Kudu > > for 2 reasons: 1) for the client which it links against (build time > > dependency) 2) for the dev environment (dev and test dependency). > > > > What are you trying to do? If you want Impala to work on ppc64le, then > > you'll probably eventually want Kudu to work on ppc64le as well, and > > that might take a lot more work/testing. > > > > If you want the impala that you build to be able to communicate with > > Kudu at all (whether or not Kudu runs on the same platform), you do > > need to compile in the client as you're currently doing. In that case, > > you should ask the Kudu authors at [email protected]. > > > > If you're OK with not supporting any Kudu for now, then you can > > "disable" Kudu support in the build/runtime. To do this, you should: > > 1) in native-toolchain: > > get source/kudu/build.sh is_supported_platform() to return false > > for ppc64le > > that results in a stub being built instead of all of kudu > > 2) in Impala: changing impala-config.sh so IS_KUDU_SUPPORTED is false > > for ppc64le. > > > > On Wed, Sep 21, 2016 at 10:45 PM, Valencia Serrao > wrote: > >> > >> Hi All, > >> > >> I am building Impala-toolchain on ppc64le. However, while building > Kudu's > >> thirdparty "libstdc++-v3", I have encountered following errors: > >> Error1: clang-3.8: error: unsupported option '-print-multi-os-directory' > >> Error2: clang-3.8: error: unknown argument: '-mlong-double-64' > >> > >> I couldn't find any documentation to find the equivalent options to be > used > >> on ppc64le. To get around the Error1, , I have removed the > >> -print-multi-os-directory option and continued with the build. > >> > >> The build has surely progressed with this work-around, however, I need > to > >> know: > >> 1. What is the impact of removing "-print-multi-os-directory" ? Will it > >> significantly affect the functionality for Impala ? > >> 2. Can I take the same approach for the Error2 ? What will be the > impact on > >> functionality ? > >> 3. If removing the above two options has a significant impact, then, > could > >> you share the documentation/equivalent options to be used for ppc64le ? > >> > >> Any pointers on this issue will be helpful. > >> > >> Regards, > >> Valencia > -- Todd Lipcon Software Engineer, Cloudera
Re: Impala Toolchain Build: Unsupported options error
I should add that the last option about "disabling" kudu we've found to be painful in practice and may cause you to have some issues running/testing. If it's possible to move forward with the toolchain compilation I would strongly recommend that. On Thu, Sep 22, 2016 at 9:13 AM, Matthew Jacobs wrote: > Any modifications you make to the Kudu build would probably need to be > checked by the folks working on the Kudu project. Impala builds Kudu > for 2 reasons: 1) for the client which it links against (build time > dependency) 2) for the dev environment (dev and test dependency). > > What are you trying to do? If you want Impala to work on ppc64le, then > you'll probably eventually want Kudu to work on ppc64le as well, and > that might take a lot more work/testing. > > If you want the impala that you build to be able to communicate with > Kudu at all (whether or not Kudu runs on the same platform), you do > need to compile in the client as you're currently doing. In that case, > you should ask the Kudu authors at [email protected]. > > If you're OK with not supporting any Kudu for now, then you can > "disable" Kudu support in the build/runtime. To do this, you should: > 1) in native-toolchain: > get source/kudu/build.sh is_supported_platform() to return false > for ppc64le > that results in a stub being built instead of all of kudu > 2) in Impala: changing impala-config.sh so IS_KUDU_SUPPORTED is false > for ppc64le. > > On Wed, Sep 21, 2016 at 10:45 PM, Valencia Serrao wrote: >> >> Hi All, >> >> I am building Impala-toolchain on ppc64le. However, while building Kudu's >> thirdparty "libstdc++-v3", I have encountered following errors: >> Error1: clang-3.8: error: unsupported option '-print-multi-os-directory' >> Error2: clang-3.8: error: unknown argument: '-mlong-double-64' >> >> I couldn't find any documentation to find the equivalent options to be used >> on ppc64le. To get around the Error1, , I have removed the >> -print-multi-os-directory option and continued with the build. >> >> The build has surely progressed with this work-around, however, I need to >> know: >> 1. What is the impact of removing "-print-multi-os-directory" ? Will it >> significantly affect the functionality for Impala ? >> 2. Can I take the same approach for the Error2 ? What will be the impact on >> functionality ? >> 3. If removing the above two options has a significant impact, then, could >> you share the documentation/equivalent options to be used for ppc64le ? >> >> Any pointers on this issue will be helpful. >> >> Regards, >> Valencia
Re: Impala Toolchain Build: Unsupported options error
Any modifications you make to the Kudu build would probably need to be checked by the folks working on the Kudu project. Impala builds Kudu for 2 reasons: 1) for the client which it links against (build time dependency) 2) for the dev environment (dev and test dependency). What are you trying to do? If you want Impala to work on ppc64le, then you'll probably eventually want Kudu to work on ppc64le as well, and that might take a lot more work/testing. If you want the impala that you build to be able to communicate with Kudu at all (whether or not Kudu runs on the same platform), you do need to compile in the client as you're currently doing. In that case, you should ask the Kudu authors at [email protected]. If you're OK with not supporting any Kudu for now, then you can "disable" Kudu support in the build/runtime. To do this, you should: 1) in native-toolchain: get source/kudu/build.sh is_supported_platform() to return false for ppc64le that results in a stub being built instead of all of kudu 2) in Impala: changing impala-config.sh so IS_KUDU_SUPPORTED is false for ppc64le. On Wed, Sep 21, 2016 at 10:45 PM, Valencia Serrao wrote: > > Hi All, > > I am building Impala-toolchain on ppc64le. However, while building Kudu's > thirdparty "libstdc++-v3", I have encountered following errors: > Error1: clang-3.8: error: unsupported option '-print-multi-os-directory' > Error2: clang-3.8: error: unknown argument: '-mlong-double-64' > > I couldn't find any documentation to find the equivalent options to be used > on ppc64le. To get around the Error1, , I have removed the > -print-multi-os-directory option and continued with the build. > > The build has surely progressed with this work-around, however, I need to > know: > 1. What is the impact of removing "-print-multi-os-directory" ? Will it > significantly affect the functionality for Impala ? > 2. Can I take the same approach for the Error2 ? What will be the impact on > functionality ? > 3. If removing the above two options has a significant impact, then, could > you share the documentation/equivalent options to be used for ppc64le ? > > Any pointers on this issue will be helpful. > > Regards, > Valencia
