Re: Required help in apache drill

[Paul Rogers]

This is Base64 encoding. You an write a UDF (user defined function) to 
implement this.

Thanks,

- Paul

 

On Friday, April 20, 2018, 10:48:26 AM PDT, Kunal Khatua  
wrote:  
 
 First, you're posting on the wrong mailing list. This should be posted in the 
user mailing list, because this is meant for discussion of development features 
about Drill. As a result, I'll cc to the dev list (so that you don't miss the 
reply), but you should carry on the conversation in the User list.

Now, coming back to your question, the encoded values you shared could be 
anything. What you seem to be looking for is a way to decipher what the bytes 
that appear as  "MTExNDUwMjM3OA=="  actually mean. This is tricky, if not 
impossible. Do you know what is the source of this data? That might be carrying 
information about the encoding. 

If not, there might be online tools that can (perhaps) decode the bytes 
correctly. 

Drill provides that capability with the convert_to function : 
https://drill.apache.org/docs/supported-data-types/#data-types-for-convert_to-and-convert_from-functions
 , but it seems you've already tried that.

At this point, your best bet is to find the encoding of the source from where 
you got this data.
On 4/20/2018 8:46:56 AM, Lavanya Nuthalapati  wrote:
Hi Team,

I have data with the encoded values "MTExNDUwMjM3OA==" , i wanted to decode
it.
Have tried convert_from and cast to varchar, nothing is working out.

Can you please help on the same.

It is very urgent your response is appreciated.

Regards,
Lavanya.  

Re: Required help in apache drill

[Kunal Khatua]

>> That's base 64 encoded data which turns out to be '1114502378'. 

Wow! That's some super power!  :D


On 4/20/2018 11:17:40 AM, Bob Rudis  wrote:
That's base 64 encoded data which turns out to be '1114502378'.

I cld have sworn Drill has built-in base64 conversion but I also wrote a UDF a 
while back that has a simple decoding function to turn base64 to text (which is 
dangerous since base64 content could and often is binary).

https://gist.github.com/hrbrmstr/ce2c49c7d4b7027437728192e278781c has an 
outline of the code.

Alternately, you can perform whatever whittling down queries you need to 
(unless you need to whittle down on the base64 encoded column) then yank the 
data into R, Python, Go, whatever and do the base 64 decoding there.

> On Apr 20, 2018, at 1:47 PM, Kunal Khatua wrote:
>
> First, you're posting on the wrong mailing list. This should be posted in the 
> user mailing list, because this is meant for discussion of development 
> features about Drill. As a result, I'll cc to the dev list (so that you don't 
> miss the reply), but you should carry on the conversation in the User list.
>
> Now, coming back to your question, the encoded values you shared could be 
> anything. What you seem to be looking for is a way to decipher what the bytes 
> that appear as "MTExNDUwMjM3OA==" actually mean. This is tricky, if not 
> impossible. Do you know what is the source of this data? That might be 
> carrying information about the encoding.
>
> If not, there might be online tools that can (perhaps) decode the bytes 
> correctly.
>
> Drill provides that capability with the convert_to function : 
> https://drill.apache.org/docs/supported-data-types/#data-types-for-convert_to-and-convert_from-functions
>  , but it seems you've already tried that.
>
> At this point, your best bet is to find the encoding of the source from where 
> you got this data.
> On 4/20/2018 8:46:56 AM, Lavanya Nuthalapati wrote:
> Hi Team,
>
> I have data with the encoded values "MTExNDUwMjM3OA==" , i wanted to decode
> it.
> Have tried convert_from and cast to varchar, nothing is working out.
>
> Can you please help on the same.
>
> It is very urgent your response is appreciated.
>
> Regards,
> Lavanya.

Re: Required help in apache drill

[Bob Rudis]

That's base 64 encoded data which turns out to be '1114502378'.

I cld have sworn Drill has built-in base64 conversion but I also wrote a UDF a 
while back that has a simple decoding function to turn base64 to text (which is 
dangerous since base64 content could and often is binary).

https://gist.github.com/hrbrmstr/ce2c49c7d4b7027437728192e278781c has an 
outline of the code.

Alternately, you can perform whatever whittling down queries you need to 
(unless you need to whittle down on the base64 encoded column) then yank the 
data into R, Python, Go, whatever and do the base 64 decoding there.

> On Apr 20, 2018, at 1:47 PM, Kunal Khatua  wrote:
> 
> First, you're posting on the wrong mailing list. This should be posted in the 
> user mailing list, because this is meant for discussion of development 
> features about Drill. As a result, I'll cc to the dev list (so that you don't 
> miss the reply), but you should carry on the conversation in the User list.
> 
> Now, coming back to your question, the encoded values you shared could be 
> anything. What you seem to be looking for is a way to decipher what the bytes 
> that appear as  "MTExNDUwMjM3OA=="  actually mean. This is tricky, if not 
> impossible. Do you know what is the source of this data? That might be 
> carrying information about the encoding.
> 
> If not, there might be online tools that can (perhaps) decode the bytes 
> correctly.
> 
> Drill provides that capability with the convert_to function : 
> https://drill.apache.org/docs/supported-data-types/#data-types-for-convert_to-and-convert_from-functions
>  , but it seems you've already tried that.
> 
> At this point, your best bet is to find the encoding of the source from where 
> you got this data.
> On 4/20/2018 8:46:56 AM, Lavanya Nuthalapati  wrote:
> Hi Team,
> 
> I have data with the encoded values "MTExNDUwMjM3OA==" , i wanted to decode
> it.
> Have tried convert_from and cast to varchar, nothing is working out.
> 
> Can you please help on the same.
> 
> It is very urgent your response is appreciated.
> 
> Regards,
> Lavanya.



signature.asc
Description: Message signed with OpenPGP

Re: Required help in apache drill

[Kunal Khatua]

First, you're posting on the wrong mailing list. This should be posted in the 
user mailing list, because this is meant for discussion of development features 
about Drill. As a result, I'll cc to the dev list (so that you don't miss the 
reply), but you should carry on the conversation in the User list.

Now, coming back to your question, the encoded values you shared could be 
anything. What you seem to be looking for is a way to decipher what the bytes 
that appear as  "MTExNDUwMjM3OA=="  actually mean. This is tricky, if not 
impossible. Do you know what is the source of this data? That might be carrying 
information about the encoding. 

If not, there might be online tools that can (perhaps) decode the bytes 
correctly. 

Drill provides that capability with the convert_to function : 
https://drill.apache.org/docs/supported-data-types/#data-types-for-convert_to-and-convert_from-functions
 , but it seems you've already tried that.

At this point, your best bet is to find the encoding of the source from where 
you got this data.
On 4/20/2018 8:46:56 AM, Lavanya Nuthalapati  wrote:
Hi Team,

I have data with the encoded values "MTExNDUwMjM3OA==" , i wanted to decode
it.
Have tried convert_from and cast to varchar, nothing is working out.

Can you please help on the same.

It is very urgent your response is appreciated.

Regards,
Lavanya.

Re: Drill on Windows

[Timothy Farkas]

Thanks everyone,

I made a ticket here https://issues.apache.org/jira/browse/DRILL-6346 . Please 
follow it to get the latest info.

Tim




From: Chris Cunningham 
Sent: Friday, April 20, 2018 7:41:48 AM
To: user@drill.apache.org
Subject: Re: Drill on Windows

Hi Tim,
I would appreciate this as well.  Thank you for offering.
-Chris

On Fri, Apr 20, 2018 at 4:54 AM, Rahul Raj 
wrote:

> Thanks for the inputs.
>
> Tim, would really appreciate if you could get the docker image done.
>
> Regards,
> Rahul
>
> On Fri, Apr 20, 2018 at 9:43 AM, scott  wrote:
>
> > Tim,
> >
> > As it turns out, I was just out there looking for an official docker
> image
> > today, and was a little disappointed there wasn't one. It's not that hard
> > to make my own, but always preferable to me to use an official image
> where
> > I can just control it using parameters.
> >
> > Would be much appreciated if you built one.
> >
> >
> > Scott
> >
> >
> >
> > On 04/19/2018 11:21 AM, Timothy Farkas wrote:
> >
> >> Rahul / Chris,
> >>
> >> The easiest way to do this would be to create a docker container for
> >> Drill. Also it looks like Docker leverage's Windows 10's linux
> subsystem so
> >> docker images don't run in a VM 
> >> https://urldefense.proofpoint.com/v2/url?u=https-3A__www.hanselman.com_blog=DwIBaQ=cskdkSMqhcnjZxdQVpwTXg=4eQVr8zB8ZBff-yxTimdOQ=DIQzoSbv5xeatvKHpNj9ly2AxFYhVDfJTqqfPZksg6w=3egN79Un6ZdskTbq_OKxqdFG9GcXJNpspPLGbtV0pj8=
> >> /DockerAndLinuxContainersOnWindowsWithOrWithoutHyperVVirtualM
> achines.aspx.
> >> So you get support for running on Windows 10 for free with no
> performance
> >> penalty. Plus everyone expects software to be package in docker images
> >> these days anyway.
> >>
> >> Currently, there is no official drill docker image. If you're
> interested,
> >> let me know, and I can create a Jira and make one. I'm pretty bored
> right
> >> now and need something to do anyway :).
> >>
> >> Thanks,
> >> Tim
> >>
> >> ___
>

Re: Drill on Windows

[Chris Cunningham]

Hi Tim,
I would appreciate this as well.  Thank you for offering.
-Chris

On Fri, Apr 20, 2018 at 4:54 AM, Rahul Raj 
wrote:

> Thanks for the inputs.
>
> Tim, would really appreciate if you could get the docker image done.
>
> Regards,
> Rahul
>
> On Fri, Apr 20, 2018 at 9:43 AM, scott  wrote:
>
> > Tim,
> >
> > As it turns out, I was just out there looking for an official docker
> image
> > today, and was a little disappointed there wasn't one. It's not that hard
> > to make my own, but always preferable to me to use an official image
> where
> > I can just control it using parameters.
> >
> > Would be much appreciated if you built one.
> >
> >
> > Scott
> >
> >
> >
> > On 04/19/2018 11:21 AM, Timothy Farkas wrote:
> >
> >> Rahul / Chris,
> >>
> >> The easiest way to do this would be to create a docker container for
> >> Drill. Also it looks like Docker leverage's Windows 10's linux
> subsystem so
> >> docker images don't run in a VM https://www.hanselman.com/blog
> >> /DockerAndLinuxContainersOnWindowsWithOrWithoutHyperVVirtualM
> achines.aspx.
> >> So you get support for running on Windows 10 for free with no
> performance
> >> penalty. Plus everyone expects software to be package in docker images
> >> these days anyway.
> >>
> >> Currently, there is no official drill docker image. If you're
> interested,
> >> let me know, and I can create a Jira and make one. I'm pretty bored
> right
> >> now and need something to do anyway :).
> >>
> >> Thanks,
> >> Tim
> >>
> >> ___
>

Re: Drill on Windows

[Rahul Raj]

Thanks for the inputs.

Tim, would really appreciate if you could get the docker image done.

Regards,
Rahul

On Fri, Apr 20, 2018 at 9:43 AM, scott  wrote:

> Tim,
>
> As it turns out, I was just out there looking for an official docker image
> today, and was a little disappointed there wasn't one. It's not that hard
> to make my own, but always preferable to me to use an official image where
> I can just control it using parameters.
>
> Would be much appreciated if you built one.
>
>
> Scott
>
>
>
> On 04/19/2018 11:21 AM, Timothy Farkas wrote:
>
>> Rahul / Chris,
>>
>> The easiest way to do this would be to create a docker container for
>> Drill. Also it looks like Docker leverage's Windows 10's linux subsystem so
>> docker images don't run in a VM https://www.hanselman.com/blog
>> /DockerAndLinuxContainersOnWindowsWithOrWithoutHyperVVirtualMachines.aspx.
>> So you get support for running on Windows 10 for free with no performance
>> penalty. Plus everyone expects software to be package in docker images
>> these days anyway.
>>
>> Currently, there is no official drill docker image. If you're interested,
>> let me know, and I can create a Jira and make one. I'm pretty bored right
>> now and need something to do anyway :).
>>
>> Thanks,
>> Tim
>>
>> 
>> From: Chris Cunningham 
>> Sent: Wednesday, April 18, 2018 4:41:13 PM
>> To: user@drill.apache.org
>> Subject: Re: Drill on Windows
>>
>> Supposedly with Windows 10, there is a full Linux subsystem that you can
>> use.  Note: I haven't done this myself, so I'm not sure what would be
>> needed (or exactly how it would work).  But it looks like a reasonable
>> option to pursue.
>>
>> https://urldefense.proofpoint.com/v2/url?u=https-3A__www.how
>> togeek.com_249966_how-2Dto-2Dinstall-2Dand-2Duse-2Dthe-2Dlin
>> ux-2Dbash-2Dshell-2Don-2Dwindows-2D10_=DwIBaQ=cskdkSMqhc
>> njZxdQVpwTXg=4eQVr8zB8ZBff-yxTimdOQ=WogK4xioUxlrOsNBOGo
>> B6W15z3N-mtPAV2Wh10p4gDU=JuU7VctfFIZOo70T7ofeMNbHrZ1U-l3FMAFevLD_DpE=
>>
>> This link says background processes aren't possible (which would make this
>> pointless for drill, right?), but it seems that has been fixed.
>>
>> On Wed, Apr 18, 2018 at 10:45 AM, Kunal Khatua  wrote:
>>
>> I might be wrong, but I think it's partly because of the need for
>>> Zookeeper on Windows, which is not commonly done.
>>>
>>> Other more obvious factors, IMO, are the overhead in creating and
>>> maintaining *nix shell scripts in Batch files. Linux (Bash) scripts are
>>> much more powerful with a lot of capabilities to make use of a rich
>>> variety
>>> of utilities that the Linux OS offers. I've worked with Windows batch
>>> files
>>> nearly a decade ago and found it handicapping. Running in Cygwin is one
>>> low-risk workaround, but no one seems to have done (or shared the
>>> process).
>>>
>>>
>>> On 4/18/2018 4:41:11 AM, Rahul Raj 
>>> wrote:
>>> Is there any reason why Drill does not run on Windows as standalone? I
>>> can
>>> only see a windows batch file for sqlline.
>>>
>>> Will it not work if we get the shell scripts translated to windows as
>>> cmd/batch files?
>>>
>>> Regards,
>>> Rahul
>>>
>>> --
>>> _*** This email and any files transmitted with it are confidential and
>>> intended solely for the use of the individual or entity to whom it is
>>> addressed. If you are not the named addressee then you should not
>>> disseminate, distribute or copy this e-mail. Please notify the sender
>>> immediately and delete this e-mail from your system.***_
>>>
>>>
>

-- 
_*** This email and any files transmitted with it are confidential and 
intended solely for the use of the individual or entity to whom it is 
addressed. If you are not the named addressee then you should not 
disseminate, distribute or copy this e-mail. Please notify the sender 
immediately and delete this e-mail from your system.***_