Cool, interested in seeing how this works.
> On Mar 28, 2018, at 2:21 PM, Kalyan Kumar Kalvagadda <kkal...@cloudera.com>
> wrote:
>
> Steve,
>
> I found the solution for issue you reported where all the the
> auto-generated files are updated regardless of the change.
> we were not alone on this. This issue is seen because of the date that was
> added in the auto generated files.
>
> Using a thrift option "generated_annotations=undated" we can avoid the date
> from being added in to generated files. I will make a change in master that
> will remove the date from all the thrift generated files. This will avoid
> this issue when some changes the thrift files.
>
> On Wed, Mar 28, 2018 at 1:56 PM, Stephen Moist <mo...@cloudera.com> wrote:
>
>>>> 1. All the the thrift auto generated files getting updates regardless
>> of
>>>> what you change in the thrift definition.
>>>> - I'm sure there should way be a way to avoid that by making changes
>> to
>>>> the maven plug-in. we need to look into that.
>> There isn’t any thing in thrift that does a before/after comparison or
>> only makes changes based on new/removed things. It fully generates the API
>> every time.
>>
>>> I agree with Sash that we rarely changes thrift API. Can we have a tool,
>>> such as git, to compare the new and old generated file, and automatically
>>> revert files not changed?
>> It’s an interesting idea. However, the tool would need to be able to
>> parse Thrift schema, know what it’ll be generating, parse git diff,
>> understand cosmetic non-functional changes(that we have the issue with now)
>> in the diff, run git checkout — <file> commands on the files and then more
>> importantly be able to detect the actual changes generated in files and not
>> skip those. This is far more complex than what I’d like for a solution.
>> This would be a full feature to develop.
>>
>>> On Mar 28, 2018, at 12:24 PM, Na Li <lina...@cloudera.com> wrote:
>>>
>>> Steve,
>>>
>>> I agree with Sash that we rarely changes thrift API. Can we have a tool,
>>> such as git, to compare the new and old generated file, and automatically
>>> revert files not changed? If so, then that will have benefits of both
>> sides.
>>> I suspect the thrift version is tied to other components using thrift
>> too.
>>> It is easier to only have one version of thrift in the whole system.
>>>
>>> Thanks,
>>>
>>> Lina
>>>
>>> On Wed, Mar 28, 2018 at 11:53 AM, Kalyan Kumar Kalvagadda <
>>> kkal...@cloudera.com> wrote:
>>>
>>>> Steve,
>>>>
>>>> I agree with Sasha.
>>>> I see two concerns that you raised
>>>>
>>>> 1. All the the thrift auto generated files getting updates regardless
>> of
>>>> what you change in the thrift definition.
>>>> - I'm sure there should way be a way to avoid that by making changes
>> to
>>>> the maven plug-in. we need to look into that.
>>>> 2. Using latest version of thrift.
>>>> - We need to understand what we are getting into. we need do some
>>>> diligence to understand what what does it mean to us and take a
>> call
>>>> on
>>>> that.
>>>>
>>>>
>>>> -Kalyan
>>>>
>>>>
>>>>
>>>> On Wed, Mar 28, 2018 at 9:34 AM, Stephen Moist <mo...@cloudera.com>
>> wrote:
>>>>
>>>>> I mentioned earlier that Hadoop requires protobuf to be installed. It
>> is
>>>>> a little inconvenient but it’s a 1 time setup. We can add a bit more
>>>>> documentation with all the full steps on how to install it and where to
>>>> get
>>>>> it. I’ve worked on projects before in the past where I need mysql setup
>>>> and
>>>>> running. I work on the Hadoop KMS/keyprovider and I have to modify my
>>>> JRE
>>>>> by installing the Unlimited JCE jars. So at least in my experience,
>>>> there
>>>>> has always been some external dependency required before I can start
>>>>> developing. It’s annoying, but not a problem as long as the
>> instructions
>>>>> are clear and complete.
>>>>>
>>>>>> This also may confuse IDEs people use (since there are files that
>> needs
>>>>> to be built in order to be seen.
>>>>> I’m not sure about most, but I’d expect nearly all devs to do a build
>> of
>>>>> the project as soon as they check it out. Then it will generate those
>>>>> files. The thrift generation is now part of the default mvn profile so
>>>> they
>>>>> can’t not not have it generated for a full build.
>>>>>
>>>>> As far as using 0.9.3, is there a reason as to why we’re on this
>> version?
>>>>> Can we update to a later release?
>>>>>
>>>>>> On Mar 27, 2018, at 7:51 PM, Alexander Kolbasov <ak...@cloudera.com>
>>>>> wrote:
>>>>>>
>>>>>> I have some concerns there. It is, indeed, a problem every time you
>>>> need
>>>>> to
>>>>>> do thrift changes, but so fat this was usually a rare event. Having
>>>>> thrift
>>>>>> as a requirement makes life inconvenient - for example the thrift
>>>> version
>>>>>> that I have on my laptop is 0.11.0 and I have to do something special
>>>> to
>>>>>> get outdated thrift 0.9.3. This also may confuse IDEs people use
>> (since
>>>>>> there are files that needs to be built in order to be seen.
>>>>>>
>>>>>> On the plus side all thrift changes will work automatically and people
>>>>>> wouldn't incidentally apply changes to thrift files.
>>>>>>
>>>>>> On Tue, Mar 27, 2018 at 1:34 PM, Stephen Moist <mo...@cloudera.com>
>>>>> wrote:
>>>>>>
>>>>>>> Looks like it about 5 seconds.
>>>>>>>
>>>>>>> org.apache.thrift.tools:maven-thrift-plugin
>>>>>>>
>>>>>>>> On Mar 27, 2018, at 3:27 PM, Sergio Pena <sergio.p...@cloudera.com>
>>>>>>> wrote:
>>>>>>>>
>>>>>>>> Thanks Stephen for stepping into this issue. I've seen this issue
>>>>> before
>>>>>>>> and it is tedious to manually roll back each file. But because I
>>>> don't
>>>>>>>> generate thrift sources too often, then I didn't care at that time.
>>>>>>>>
>>>>>>>> Anyway, thrift apis are in source control to the time generating
>> them
>>>>> in
>>>>>>>> every compile I assume.
>>>>>>>> How much extra time we'll have if we send them to the target/
>>>> directory
>>>>>>> and
>>>>>>>> generate them on every compilation?
>>>>>>>>
>>>>>>>> What maven plugin are you talking about?
>>>>>>>>
>>>>>>>> On Tue, Mar 27, 2018 at 1:59 PM, Stephen Moist <mo...@cloudera.com>
>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hey all, I’ve started writing some skeleton apis for ABAC. In the
>>>>>>>>> process, I’ve had to regenerate the Thrift api. Anytime that
>>>> happens,
>>>>>>> it
>>>>>>>>> regenerates all the files and gives it a new timestamp. So when I
>>>>> look
>>>>>>> in
>>>>>>>>> git, it shows 75+ files have changed in addition to my new set of
>>>>>>> apis. I
>>>>>>>>> then have to manually roll back each file. As a development
>> process
>>>>>>> this
>>>>>>>>> is terrible. So…
>>>>>>>>>
>>>>>>>>> 1) Is there any reason why we need thrift apis in source control?
>>>>>>>>> 2) Anyone object to using a newer maven plugin to generate the
>>>> thrift
>>>>>>> apis
>>>>>>>>> on build time?
>>>>>>>>> 2a) This requires all developers to install thrift 0.9.3. I
>>>>> personally
>>>>>>>>> don’t see a problem as I have to have protobuf installed to compile
>>>>>>>>> hadoop. (Most sentry devs probably already have it anyway)
>>>>>>>>> 2b) This will impact build systems to have thrift compiler
>>>> installed.
>>>>>>>>> 3) This prevents the above headache and makes it far simpler and
>>>>> faster
>>>>>>> to
>>>>>>>>> develop for Sentry. And makes for happier devs.
>>>>>>>>> 4) Dumping generated code into target/generated-sources is a
>>>> standard
>>>>>>>>> development practice when working against a known schema.
>>>>>>>
>>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> *Kalyan Kumar Kalvagadda* | Software Engineer
>>>> t. (469) 279- <0000000000>5732
>>>> cloudera.com <https://www.cloudera.com>
>>>>
>>>> [image: Cloudera] <https://www.cloudera.com/>
>>>>
>>>> [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
>>>> Cloudera on Facebook] <https://www.facebook.com/cloudera> [image:
>> Cloudera
>>>> on LinkedIn] <https://www.linkedin.com/company/cloudera>
>>>> ------------------------------
>>>>
>>
>>
>
>
> --
> *Kalyan Kumar Kalvagadda* | Software Engineer
> t. (469) 279- <0000000000>5732
> cloudera.com <https://www.cloudera.com>
>
> [image: Cloudera] <https://www.cloudera.com/>
>
> [image: Cloudera on Twitter] <https://twitter.com/cloudera> [image:
> Cloudera on Facebook] <https://www.facebook.com/cloudera> [image: Cloudera
> on LinkedIn] <https://www.linkedin.com/company/cloudera>
> ------------------------------
