Re: How to measure network traffic for a query?

2018-08-17 Thread Philipp Krause
Philipp Krause Am 16.08.2018 um 23:00 schrieb Tim Armstrong: Hey,  It looks like this got neglected. Maybe Lars or Michael knows the answer? - Tim On Fri, Aug 10, 2018 at 3:53 AM, Philipp Krause <mailto:philippkrause.m...@googlemail.com>> wrote: Hi! I would like to measure th

Re: Local join instead of data exchange - co-located blocks

2018-05-14 Thread Philipp Krause
Hello Alex, I suppose you're very busy, so I apologize for the interruption. If you have any idea of what I could try to solve this problem, please let me know. Currently I don't know how to progress and I'd appreciate any help you can give me. Best regards Philipp Philipp Krause

Re: Local join instead of data exchange - co-located blocks

2018-04-15 Thread Philipp Krause
13, 2018 at 11:00 AM, Philipp Krause <philippkrause.m...@googlemail.com <mailto:philippkrause.m...@googlemail.com>> wrote: Yes, I have a running (virtual) cluster. I would try to follow your way with the custom impala build (DistributedPlanner.java is the only modified file at t

Re: Local join instead of data exchange - co-located blocks

2018-04-13 Thread Philipp Krause
Managed cluster, you can deploy > a custom Impala build by simply overwriting the Impala existing binaries > and jars with the new build. If you want to go this route, I can give you a > full list of files you need to replace. > > On Tue, Apr 10, 2018 at 11:44 AM, Philipp Krause &

Re: Local join instead of data exchange - co-located blocks

2018-03-28 Thread Philipp Krause
ntric. For each host, collect the local scan ranges of *all* scan nodes, and assign them to that host. Does that make sense? Alex On Mon, Mar 19, 2018 at 1:02 PM, Philipp Krause <philippkrause.m...@googlemail.com <mailto:philippkrause.m...@googlemail.com>> wrote: I'd like t

Re: Local join instead of data exchange - co-located blocks

2018-03-19 Thread Philipp Krause
very much for your time! Philipp Am 18.03.2018 um 17:32 schrieb Philipp Krause: Hi! At the moment the data to parquet (block) mapping is based on a simple modulo function: Id % #data_nodes. So with 5 data nodes all rows with Id's 0,5,10,... are written to Parquet_0, Id's 1,4,9 are written to

Re: Local join instead of data exchange - co-located blocks

2018-03-18 Thread Philipp Krause
uld be really grateful if you could give me a hint for the scheduling implementation. I try to go deeper through the code meanwhile. Best regards and thank you in advance Philipp Am 14.03.2018 um 08:06 schrieb Philipp Krause: Thank you very much for these information! I'll try to implement the

Re: Local join instead of data exchange - co-located blocks

2018-03-14 Thread Philipp Krause
gt; > You'll need to dig through and understand that code so that you can make > the necessary changes. Change the scan range to host mapping to your > liking. The rest of the code should just work. > > Cheers, > > Alex > > > On Mon, Mar 12, 2018 at 6:55 PM, Philipp Krause <

Re: Local join instead of data exchange - co-located blocks

2018-03-12 Thread Philipp Krause
x On Mon, Mar 12, 2018 at 6:29 AM, Philipp Krause <philippkrause.m...@googlemail.com <mailto:philippkrause.m...@googlemail.com>> wrote: Hello everyone! In order to prevent network traffic, I'd like to perform local joins on each node instead of ex

Local join instead of data exchange - co-located blocks

2018-03-12 Thread Philipp Krause
regards Philipp Krause