[jira] [Commented] (JOSHUA-251) Address Website Branding Issues

[Matt Post (JIRA)]

[ 
https://issues.apache.org/jira/browse/JOSHUA-251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15261475#comment-15261475
 ] 

Matt Post commented on JOSHUA-251:
--

Related to this: after trying to use it a little, I think the current 
website-building approach is too much of an impediment to use. Recall that this 
process requires maintaining two branches in the repo: one for the source files 
(in Markdown format, mostly), and another for the generated website, which then 
gets pushed up. When the website was on Github, we just needed the source 
branch, because Github runs Jekyll for you. 

Another nice feature of Github was you could easily edit the files in a web 
browser on the site directly, and that would also trigger an update. It'd be 
nice to remove any barriers to documentation, since it's already kind of hard 
to get done.

I'm thinking about moving the website over to Joshua's Confluence page. Are 
there any drawbacks to this? I'm a little wary of putting the site in a 
proprietary CMS, but it seems that Apache is all-in on the software, and it 
provides a good user experience.

matt

> Address Website Branding Issues
> ---
>
> Key: JOSHUA-251
> URL: https://issues.apache.org/jira/browse/JOSHUA-251
> Project: Joshua
>  Issue Type: Task
>Reporter: Lewis John McGibbney
>Priority: Critical
> Fix For: 6.1
>
>
> We have a number of Website branding issues which we need to address.
> http://www.apache.org/foundation/marks/pmcs.html#introduction
> Lets work through them here. Please create child issues if appropriate.
> Thanks



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-joshua pull request: More work on structuring translatio...

[mjpost]

Github user mjpost commented on the pull request:

https://github.com/apache/incubator-joshua/pull/6#issuecomment-215286087
  
Okay, going to hold off until I hear from you...


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-joshua pull request: More work on structuring translatio...

[mjpost]

Github user mjpost commented on the pull request:

https://github.com/apache/incubator-joshua/pull/6#issuecomment-215274644
  
Should I wait then to pull this in and fix the test cases?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: joshua_api

[Matt Post]

Sure thing, hope to tonight.

matt


> On Apr 27, 2016, at 6:41 PM, kellen sunderland  
> wrote:
> 
> Hey Matt,
> 
> If you had time that would be fantastic.  I've created a new PR in case you
> want to pull it in.  There's actually 4 tests failing for me currently
> (casing issues causing at least one).  If you want to wait until we fix
> these tests that's also completely fine.
> 
> -Kellen
> 
> On Wed, Apr 27, 2016 at 11:32 AM, Matt Post  wrote:
> 
>> Do you want me to fix the recapitalization? Or are you going to do that? I
>> looked a bit, and it seems I'll have to add a method to get a word
>> alignment object instead of just the string, so that I can poke through
>> them. This approach is as good as true-casing in some languages.
>> 
>> A few other things:
>> 
>> - I saw a comment in the commit about the changes not working for
>> phrase-based translation. Can you (or Felix) elaborate? What exactly will
>> no longer work?
>> 
>> - Currently, there are multiple places where the "output-format" string
>> has to get edited (KBestExtractor and in Translation). After you push your
>> changes in, I'm going to make some edits so that this all occurs in one
>> place.
>> 
>> matt
>> 
>> 
>>> On Apr 27, 2016, at 2:25 PM, kellen sunderland <
>> kellen.sunderl...@gmail.com> wrote:
>>> 
>>> Thanks for taking a look Matt,
>>> 
>>> I think this is all we've got planned as far as changes relating to an
>> API
>>> would go.  We have a few more commits coming but they're just performance
>>> improvements and they don't change too much in the way of interfaces or
>>> method signatures.
>>> 
>>> -Kellen
>>> 
>>> On Wed, Apr 27, 2016 at 4:47 AM, Matt Post  wrote:
>>> 
 Kellen,
 
 Great. I had a chance to start looking over the ReworkedExtractions
 branch. I'll have some more time today. It looks good to me so far. Is
 there anything else you plan to do, or does that branch contain
>> basically
 all of it (apart from the recapitalization fix, which I see should be
 applied more selectively, maybe only when a -recapitalize flag is
>> present,
 to save on time).
 
 matt
 
 
> On Apr 26, 2016, at 1:56 AM, kellen sunderland <
 kellen.sunderl...@gmail.com> wrote:
> 
> Hey Matt,
> 
> I've opened a new pull request with a few of our commits, feel free to
 take
> a look when you have some time.
> 
> More importantly I've pushed our queue of upcoming commits to the
 following
> branch in my fork:
> 
 
>> https://github.com/KellenSunderland/incubator-joshua/commits/ReworkedExtractions
> .  From there you can get an idea for the work we've done so far.  I
> haven't opened a PR yet for these commits because there's still some
> merging I have to do (there's a few failing tests and I had to
 temporarily
> comment out some of your casing code).  Once that's fixed I'll do a
 proper
> PR for these commits.
> 
> -Kellen
> 
> On Mon, Apr 25, 2016 at 1:35 PM, Matt Post  wrote:
> 
>> Great. On that first point, I meant that translate() would return a
>> Translation object, which would know its hypergraph and could iterate
 over
>> a KBestExtractor. In any case, though, it sounds like you are a bit
 ahead
>> of me on this, so I'll wait for a push that I can see, and then we can
>> converge on the design.
>> 
>> matt
>> 
>> 
>>> On Apr 25, 2016, at 4:10 PM, Hieber, Felix 
>> wrote:
>>> 
>>> Hi Matt,
>>> 
>>> These are some nice suggestions. Most of the work we have done is in
>> line of what you propose so I would agree with Kellen that we should
>> synchronize and compare better earlier than later.
>>> 
>>> Best,
>>> Felix
>>> 
 On 25.04.2016, at 07:44, kellen sunderland <
 kellen.sunderl...@gmail.com>
>> wrote:
 
 Hey Matt,
 
 Sorry for the late reply.  The Joshua-6 folder and tst may have just
>> been
 artifacts of some symlinks I have locally.  Sorry they may have been
>> pushed
 by mistake, I can clean that up.
 
 Good idea to have the api code in a separate branch.  We can merge
>> the
>> work
 that we've done some time next week.
 
 KBestExtractor is one of the things we want to return via the API.
>> We
 already have some of this implemented though as you suggest.  I'll
>> try
>> and
 push the remaining work we've done into my github branch so you can
>> compare.
 
 -Kellen
 
> On Mon, Apr 25, 2016 at 6:11 AM, Matt Post 
>> wrote:
> 
> Okay, after looking at this a bit more, I have a better
 understanding,
>> and
> an idea for how to move forward.
> 
> First, I see 

[GitHub] incubator-joshua pull request: More work on structuring translatio...

[KellenSunderland]

Github user KellenSunderland commented on the pull request:

https://github.com/apache/incubator-joshua/pull/6#issuecomment-215274462
  
Closing this PR temporarily as we want to tweak some commit messages.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] incubator-joshua pull request: More work on structuring translatio...

[KellenSunderland]

Github user KellenSunderland closed the pull request at:

https://github.com/apache/incubator-joshua/pull/6


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: joshua_api

[kellen sunderland]

Hey Matt,

If you had time that would be fantastic.  I've created a new PR in case you
want to pull it in.  There's actually 4 tests failing for me currently
(casing issues causing at least one).  If you want to wait until we fix
these tests that's also completely fine.

-Kellen

On Wed, Apr 27, 2016 at 11:32 AM, Matt Post  wrote:

> Do you want me to fix the recapitalization? Or are you going to do that? I
> looked a bit, and it seems I'll have to add a method to get a word
> alignment object instead of just the string, so that I can poke through
> them. This approach is as good as true-casing in some languages.
>
> A few other things:
>
> - I saw a comment in the commit about the changes not working for
> phrase-based translation. Can you (or Felix) elaborate? What exactly will
> no longer work?
>
> - Currently, there are multiple places where the "output-format" string
> has to get edited (KBestExtractor and in Translation). After you push your
> changes in, I'm going to make some edits so that this all occurs in one
> place.
>
> matt
>
>
> > On Apr 27, 2016, at 2:25 PM, kellen sunderland <
> kellen.sunderl...@gmail.com> wrote:
> >
> > Thanks for taking a look Matt,
> >
> > I think this is all we've got planned as far as changes relating to an
> API
> > would go.  We have a few more commits coming but they're just performance
> > improvements and they don't change too much in the way of interfaces or
> > method signatures.
> >
> > -Kellen
> >
> > On Wed, Apr 27, 2016 at 4:47 AM, Matt Post  wrote:
> >
> >> Kellen,
> >>
> >> Great. I had a chance to start looking over the ReworkedExtractions
> >> branch. I'll have some more time today. It looks good to me so far. Is
> >> there anything else you plan to do, or does that branch contain
> basically
> >> all of it (apart from the recapitalization fix, which I see should be
> >> applied more selectively, maybe only when a -recapitalize flag is
> present,
> >> to save on time).
> >>
> >> matt
> >>
> >>
> >>> On Apr 26, 2016, at 1:56 AM, kellen sunderland <
> >> kellen.sunderl...@gmail.com> wrote:
> >>>
> >>> Hey Matt,
> >>>
> >>> I've opened a new pull request with a few of our commits, feel free to
> >> take
> >>> a look when you have some time.
> >>>
> >>> More importantly I've pushed our queue of upcoming commits to the
> >> following
> >>> branch in my fork:
> >>>
> >>
> https://github.com/KellenSunderland/incubator-joshua/commits/ReworkedExtractions
> >>> .  From there you can get an idea for the work we've done so far.  I
> >>> haven't opened a PR yet for these commits because there's still some
> >>> merging I have to do (there's a few failing tests and I had to
> >> temporarily
> >>> comment out some of your casing code).  Once that's fixed I'll do a
> >> proper
> >>> PR for these commits.
> >>>
> >>> -Kellen
> >>>
> >>> On Mon, Apr 25, 2016 at 1:35 PM, Matt Post  wrote:
> >>>
>  Great. On that first point, I meant that translate() would return a
>  Translation object, which would know its hypergraph and could iterate
> >> over
>  a KBestExtractor. In any case, though, it sounds like you are a bit
> >> ahead
>  of me on this, so I'll wait for a push that I can see, and then we can
>  converge on the design.
> 
>  matt
> 
> 
> > On Apr 25, 2016, at 4:10 PM, Hieber, Felix 
> wrote:
> >
> > Hi Matt,
> >
> > These are some nice suggestions. Most of the work we have done is in
>  line of what you propose so I would agree with Kellen that we should
>  synchronize and compare better earlier than later.
> >
> > Best,
> > Felix
> >
> >> On 25.04.2016, at 07:44, kellen sunderland <
> >> kellen.sunderl...@gmail.com>
>  wrote:
> >>
> >> Hey Matt,
> >>
> >> Sorry for the late reply.  The Joshua-6 folder and tst may have just
>  been
> >> artifacts of some symlinks I have locally.  Sorry they may have been
>  pushed
> >> by mistake, I can clean that up.
> >>
> >> Good idea to have the api code in a separate branch.  We can merge
> the
>  work
> >> that we've done some time next week.
> >>
> >> KBestExtractor is one of the things we want to return via the API.
> We
> >> already have some of this implemented though as you suggest.  I'll
> try
>  and
> >> push the remaining work we've done into my github branch so you can
>  compare.
> >>
> >> -Kellen
> >>
> >>> On Mon, Apr 25, 2016 at 6:11 AM, Matt Post 
> wrote:
> >>>
> >>> Okay, after looking at this a bit more, I have a better
> >> understanding,
>  and
> >>> an idea for how to move forward.
> >>>
> >>> First, I see that Translation.java has provisions for structured
>  output.
> >>> I'm guessing StructuredTranslation was added by mistake?
> >>>
> >>> Moving forward, on the joshua_api branch, I was thinking of the
> 

[GitHub] incubator-joshua pull request: More work on structuring translatio...

[KellenSunderland]

GitHub user KellenSunderland opened a pull request:

https://github.com/apache/incubator-joshua/pull/6

More work on structuring translation output

These commits focus on two areas:  
*  One is structured translation output, and this should be our last PR for 
this topic for the time being.  
*  We also have done some static code analysis and fixed different issues 
that have been flagged by our tool.  

There are also some new unit tests included.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/KellenSunderland/incubator-joshua master

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-joshua/pull/6.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #6


commit 53de11905134e32191d11cafd07d5a033c16e411
Author: Felix Hieber 
Date:   2015-11-25T14:11:27Z

Reworked most of the hypergraph traversals for Viterbi and n-best 
extractions. Most importantly: translation string extraction is now int based 
instead of doing regex matching and string operations. This should be a lot 
faster. However this will only work for hiero models for now. Phrase-based 
decoding will still use the String-based extractions. Before there were two 
ways to traverse the hypergraph: (1) regular tailNode order (used for Viterbi 
and WordAlignment) and (2) tailNode order according to target side non terminal 
indices (used for KBestExtraction). This caused quite some inconsistencies on 
how to write general extractors (output string, input string, feature vector, 
word alignments, tree, etc.) that support both. The main issue was that some 
extractors (String-based HypothesisExtractor) relies on the traversal order (2) 
to simply always merge children strings into the first nonTerminal symbol on 
the target side. However, this breaks the very same class when the input st
 ring is requested (which is a supported feature in Joshua). This change gets 
rid of these inconsistencies for Hiero and simplifies a lot of the code. For 
phrase-based decoding, we still rely on the HypothesisExtractor, but this is 
not a use case for Saar at the moment, and phrase-based decoding is hack in 
Joshua anyway. JoshuaConfiguration now throws an exception if you want to have 
the 'align_index' in output strings (Moses style) for Hiero models. This is not 
supported by int[]-based extraction and useless anyway.

commit c72faea7c241395b8827cf6ab41aec67c7fdc54c
Author: Pavel Danchenko 
Date:   2015-12-22T11:49:22Z

LanguageModelFF.estimateFutureCost refactorings and test

commit d1c3caac1da8c3c1175059b20c411a3ebd965465
Author: Kellen Sunderland 
Date:   2016-04-27T22:12:00Z

Updated license files for tests

commit 9c3f2e6e60e68c9a55733d872d15c5c39c937ab0
Author: Felix Hieber 
Date:   2015-12-31T10:57:18Z

Modified KenLM jni to support querying the lm using strings not only ids. 
Also added a method to check whether a word or id is known to the lm.
Made output of regression tests more concise

commit 8d86ff2b989c2b1db8aff7eaaa479cae38c73357
Author: Felix Hieber 
Date:   2016-02-03T14:17:30Z

Mostly a refactor for improved readability




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Re: [jira] [Commented] (JOSHUA-253) Enable execution of Unit tests

[Matt Post]

I am fine with you just doing this. The current setup was a 
something-is-better-than-nothing (which is true) hack, and I'd be happy to have 
better practices pushed into the project. 

matt (from my phone)

> On Apr 27, 2016, at 2:39 PM, Kellen Sunderland (JIRA)  wrote:
> 
> 
>[ 
> https://issues.apache.org/jira/browse/JOSHUA-253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15260709#comment-15260709
>  ] 
> 
> Kellen Sunderland commented on JOSHUA-253:
> --
> 
> We've got a few unit tests we've created for Joshua, and we'd like to 
> eventually hook them into the Joshua build process.  
> 
> This is one topic I'd like to discuss at ApacheCon.  What I would like to 
> propose is to convert the current regression tests to be run by a unit test 
> runner (at the same time as the actual unit tests are run).  The main 
> advantage of having the regression tests runnable from a unit test runner is 
> that we'll be able to debug when there's a failure (this is quite tricky at 
> the moment).
> 
>> Enable execution of Unit tests
>> --
>> 
>>Key: JOSHUA-253
>>URL: https://issues.apache.org/jira/browse/JOSHUA-253
>>Project: Joshua
>> Issue Type: Test
>>   Affects Versions: 6.0
>>   Reporter: Lewis John McGibbney
>>Fix For: 6.1
>> 
>> 
>> As per our [discussion on this 
>> topic|http://www.mail-archive.com/dev%40joshua.incubator.apache.org/msg00270.html],
>>  [~teofili] correctly identified that unit level tests are not executed.
>> We need to fix this such that they are.
> 
> 
> 
> --
> This message was sent by Atlassian JIRA
> (v6.3.4#6332)

Re: joshua_api

[Matt Post]

Do you want me to fix the recapitalization? Or are you going to do that? I 
looked a bit, and it seems I'll have to add a method to get a word alignment 
object instead of just the string, so that I can poke through them. This 
approach is as good as true-casing in some languages.

A few other things:

- I saw a comment in the commit about the changes not working for phrase-based 
translation. Can you (or Felix) elaborate? What exactly will no longer work?

- Currently, there are multiple places where the "output-format" string has to 
get edited (KBestExtractor and in Translation). After you push your changes in, 
I'm going to make some edits so that this all occurs in one place.

matt


> On Apr 27, 2016, at 2:25 PM, kellen sunderland  
> wrote:
> 
> Thanks for taking a look Matt,
> 
> I think this is all we've got planned as far as changes relating to an API
> would go.  We have a few more commits coming but they're just performance
> improvements and they don't change too much in the way of interfaces or
> method signatures.
> 
> -Kellen
> 
> On Wed, Apr 27, 2016 at 4:47 AM, Matt Post  wrote:
> 
>> Kellen,
>> 
>> Great. I had a chance to start looking over the ReworkedExtractions
>> branch. I'll have some more time today. It looks good to me so far. Is
>> there anything else you plan to do, or does that branch contain basically
>> all of it (apart from the recapitalization fix, which I see should be
>> applied more selectively, maybe only when a -recapitalize flag is present,
>> to save on time).
>> 
>> matt
>> 
>> 
>>> On Apr 26, 2016, at 1:56 AM, kellen sunderland <
>> kellen.sunderl...@gmail.com> wrote:
>>> 
>>> Hey Matt,
>>> 
>>> I've opened a new pull request with a few of our commits, feel free to
>> take
>>> a look when you have some time.
>>> 
>>> More importantly I've pushed our queue of upcoming commits to the
>> following
>>> branch in my fork:
>>> 
>> https://github.com/KellenSunderland/incubator-joshua/commits/ReworkedExtractions
>>> .  From there you can get an idea for the work we've done so far.  I
>>> haven't opened a PR yet for these commits because there's still some
>>> merging I have to do (there's a few failing tests and I had to
>> temporarily
>>> comment out some of your casing code).  Once that's fixed I'll do a
>> proper
>>> PR for these commits.
>>> 
>>> -Kellen
>>> 
>>> On Mon, Apr 25, 2016 at 1:35 PM, Matt Post  wrote:
>>> 
 Great. On that first point, I meant that translate() would return a
 Translation object, which would know its hypergraph and could iterate
>> over
 a KBestExtractor. In any case, though, it sounds like you are a bit
>> ahead
 of me on this, so I'll wait for a push that I can see, and then we can
 converge on the design.
 
 matt
 
 
> On Apr 25, 2016, at 4:10 PM, Hieber, Felix  wrote:
> 
> Hi Matt,
> 
> These are some nice suggestions. Most of the work we have done is in
 line of what you propose so I would agree with Kellen that we should
 synchronize and compare better earlier than later.
> 
> Best,
> Felix
> 
>> On 25.04.2016, at 07:44, kellen sunderland <
>> kellen.sunderl...@gmail.com>
 wrote:
>> 
>> Hey Matt,
>> 
>> Sorry for the late reply.  The Joshua-6 folder and tst may have just
 been
>> artifacts of some symlinks I have locally.  Sorry they may have been
 pushed
>> by mistake, I can clean that up.
>> 
>> Good idea to have the api code in a separate branch.  We can merge the
 work
>> that we've done some time next week.
>> 
>> KBestExtractor is one of the things we want to return via the API.  We
>> already have some of this implemented though as you suggest.  I'll try
 and
>> push the remaining work we've done into my github branch so you can
 compare.
>> 
>> -Kellen
>> 
>>> On Mon, Apr 25, 2016 at 6:11 AM, Matt Post  wrote:
>>> 
>>> Okay, after looking at this a bit more, I have a better
>> understanding,
 and
>>> an idea for how to move forward.
>>> 
>>> First, I see that Translation.java has provisions for structured
 output.
>>> I'm guessing StructuredTranslation was added by mistake?
>>> 
>>> Moving forward, on the joshua_api branch, I was thinking of the
 following,
>>> but want to make sure it doesn't collide with what you've done or are
 doing:
>>> 
>>> - Factor KBestExtractor to return Translation objects instead of
 printing,
>>> and also turn it into an iterator
>>> 
>>> - There's a real discrepancy with competing forest representations.
 There
>>> are operations on the hypergraph (via WalkerFunction), and then also
>>> operations on Derivations. This leads to code that operates on both.
>> It
>>> would be nice if the KBestExtractor just returned something like a
 

Re: joshua_api

[kellen sunderland]

Thanks for taking a look Matt,

I think this is all we've got planned as far as changes relating to an API
would go.  We have a few more commits coming but they're just performance
improvements and they don't change too much in the way of interfaces or
method signatures.

-Kellen

On Wed, Apr 27, 2016 at 4:47 AM, Matt Post  wrote:

> Kellen,
>
> Great. I had a chance to start looking over the ReworkedExtractions
> branch. I'll have some more time today. It looks good to me so far. Is
> there anything else you plan to do, or does that branch contain basically
> all of it (apart from the recapitalization fix, which I see should be
> applied more selectively, maybe only when a -recapitalize flag is present,
> to save on time).
>
> matt
>
>
> > On Apr 26, 2016, at 1:56 AM, kellen sunderland <
> kellen.sunderl...@gmail.com> wrote:
> >
> > Hey Matt,
> >
> > I've opened a new pull request with a few of our commits, feel free to
> take
> > a look when you have some time.
> >
> > More importantly I've pushed our queue of upcoming commits to the
> following
> > branch in my fork:
> >
> https://github.com/KellenSunderland/incubator-joshua/commits/ReworkedExtractions
> > .  From there you can get an idea for the work we've done so far.  I
> > haven't opened a PR yet for these commits because there's still some
> > merging I have to do (there's a few failing tests and I had to
> temporarily
> > comment out some of your casing code).  Once that's fixed I'll do a
> proper
> > PR for these commits.
> >
> > -Kellen
> >
> > On Mon, Apr 25, 2016 at 1:35 PM, Matt Post  wrote:
> >
> >> Great. On that first point, I meant that translate() would return a
> >> Translation object, which would know its hypergraph and could iterate
> over
> >> a KBestExtractor. In any case, though, it sounds like you are a bit
> ahead
> >> of me on this, so I'll wait for a push that I can see, and then we can
> >> converge on the design.
> >>
> >> matt
> >>
> >>
> >>> On Apr 25, 2016, at 4:10 PM, Hieber, Felix  wrote:
> >>>
> >>> Hi Matt,
> >>>
> >>> These are some nice suggestions. Most of the work we have done is in
> >> line of what you propose so I would agree with Kellen that we should
> >> synchronize and compare better earlier than later.
> >>>
> >>> Best,
> >>> Felix
> >>>
>  On 25.04.2016, at 07:44, kellen sunderland <
> kellen.sunderl...@gmail.com>
> >> wrote:
> 
>  Hey Matt,
> 
>  Sorry for the late reply.  The Joshua-6 folder and tst may have just
> >> been
>  artifacts of some symlinks I have locally.  Sorry they may have been
> >> pushed
>  by mistake, I can clean that up.
> 
>  Good idea to have the api code in a separate branch.  We can merge the
> >> work
>  that we've done some time next week.
> 
>  KBestExtractor is one of the things we want to return via the API.  We
>  already have some of this implemented though as you suggest.  I'll try
> >> and
>  push the remaining work we've done into my github branch so you can
> >> compare.
> 
>  -Kellen
> 
> > On Mon, Apr 25, 2016 at 6:11 AM, Matt Post  wrote:
> >
> > Okay, after looking at this a bit more, I have a better
> understanding,
> >> and
> > an idea for how to move forward.
> >
> > First, I see that Translation.java has provisions for structured
> >> output.
> > I'm guessing StructuredTranslation was added by mistake?
> >
> > Moving forward, on the joshua_api branch, I was thinking of the
> >> following,
> > but want to make sure it doesn't collide with what you've done or are
> >> doing:
> >
> > - Factor KBestExtractor to return Translation objects instead of
> >> printing,
> > and also turn it into an iterator
> >
> > - There's a real discrepancy with competing forest representations.
> >> There
> > are operations on the hypergraph (via WalkerFunction), and then also
> > operations on Derivations. This leads to code that operates on both.
> It
> > would be nice if the KBestExtractor just returned something like a
> >> reduced
> > "slice" of a forest forest new nodes containing only single back
> >> pointers,
> > representing exactly the nth-best derivation. Then we could
> >> generically use
> > the WalkerFunctions on that (e.g., viterbi extraction), and get rid
> of
> >> many
> > of the DerivationVisitor classes
> >
> > - Related: constructing the k-best list is expensive, even for just
> the
> > first item, since you have to set up all the candidate lists and so
> on.
> > This led to me implementing top-n = 0, where you can get the
> >> translation
> > and some limited information (not replayed features) via Viterbi
> >> extractors
> > on the hypergraph, and you only have to call KBestExtractor if you
> >> actually
> > want k-best lists. This leads to dual code, e.g., substitutions of
> > output_format in multiple 

[jira] [Commented] (JOSHUA-256) Note that Joshua builds and runs with >= Java 1.8

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/JOSHUA-256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15260657#comment-15260657
 ] 

Hudson commented on JOSHUA-256:
---

SUCCESS: Integrated in joshua_master #27 (See 
[https://builds.apache.org/job/joshua_master/27/])
JOSHUA-256 Note that Joshua builds and runs with >= Java 1.8 
(lewis.j.mcgibbney: rev 6f45d19b1f9c7bc33c57b7485d99f27bd2e2eda6)
* README.md


> Note that Joshua builds and runs with >= Java 1.8
> -
>
> Key: JOSHUA-256
> URL: https://issues.apache.org/jira/browse/JOSHUA-256
> Project: Joshua
>  Issue Type: Improvement
>Reporter: Lewis John McGibbney
>Assignee: Lewis John McGibbney
>Priority: Critical
> Fix For: 6.1
>
>
> Since the Joshua code was transitioned into the Incubator, code has been 
> committed which requires Java 1.8 minimum to compile and run. 
> We should note this within the README and also on the Website once we move 
> towards out first 6.1 Incubating release. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (JOSHUA-256) Note that Joshua builds and runs with >= Java 1.8

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/JOSHUA-256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15260529#comment-15260529
 ] 

ASF GitHub Bot commented on JOSHUA-256:
---

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-joshua/pull/5


> Note that Joshua builds and runs with >= Java 1.8
> -
>
> Key: JOSHUA-256
> URL: https://issues.apache.org/jira/browse/JOSHUA-256
> Project: Joshua
>  Issue Type: Improvement
>Reporter: Lewis John McGibbney
>Assignee: Lewis John McGibbney
>Priority: Critical
> Fix For: 6.1
>
>
> Since the Joshua code was transitioned into the Incubator, code has been 
> committed which requires Java 1.8 minimum to compile and run. 
> We should note this within the README and also on the Website once we move 
> towards out first 6.1 Incubating release. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-joshua pull request: JOSHUA-256 Note that Joshua builds ...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/incubator-joshua/pull/5


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (JOSHUA-253) Enable execution of Unit tests

[Matt Post (JIRA)]

[ 
https://issues.apache.org/jira/browse/JOSHUA-253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15260525#comment-15260525
 ] 

Matt Post commented on JOSHUA-253:
--

Yes, Tommaso is correct. test/ holds what I think are better termed regression 
tests, and executes any executable test*sh file under test, where returning 0 
is success and otherwise is failure.

I know there are unit tests scattered throughout the code but I have never run 
them. It would be great to have those start to be run as well. I know the 
Amazon folks have been contributing some, so maybe they could let us know?

> Enable execution of Unit tests
> --
>
> Key: JOSHUA-253
> URL: https://issues.apache.org/jira/browse/JOSHUA-253
> Project: Joshua
>  Issue Type: Test
>Affects Versions: 6.0
>Reporter: Lewis John McGibbney
> Fix For: 6.1
>
>
> As per our [discussion on this 
> topic|http://www.mail-archive.com/dev%40joshua.incubator.apache.org/msg00270.html],
>  [~teofili] correctly identified that unit level tests are not executed.
> We need to fix this such that they are.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-joshua pull request: JOSHUA-256 Note that Joshua builds ...

[lewismc]

GitHub user lewismc opened a pull request:

https://github.com/apache/incubator-joshua/pull/5

JOSHUA-256 Note that Joshua builds and runs with >= Java 1.8

This PR addresses https://issues.apache.org/jira/browse/JOSHUA-256

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/lewismc/incubator-joshua JOSHUA-256

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-joshua/pull/5.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #5


commit 6f45d19b1f9c7bc33c57b7485d99f27bd2e2eda6
Author: Lewis John McGibbney 
Date:   2016-04-27T15:23:31Z

JOSHUA-256 Note that Joshua builds and runs with >= Java 1.8




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (JOSHUA-253) Enable execution of Unit tests

[Lewis John McGibbney (JIRA)]

[ 
https://issues.apache.org/jira/browse/JOSHUA-253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15260269#comment-15260269
 ] 

Lewis John McGibbney commented on JOSHUA-253:
-

bq. so no Java unit test is executed as far as I know

where are the unit tests? I can fix this right now if you can point them out. 
Thanks

> Enable execution of Unit tests
> --
>
> Key: JOSHUA-253
> URL: https://issues.apache.org/jira/browse/JOSHUA-253
> Project: Joshua
>  Issue Type: Test
>Affects Versions: 6.0
>Reporter: Lewis John McGibbney
> Fix For: 6.1
>
>
> As per our [discussion on this 
> topic|http://www.mail-archive.com/dev%40joshua.incubator.apache.org/msg00270.html],
>  [~teofili] correctly identified that unit level tests are not executed.
> We need to fix this such that they are.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (JOSHUA-253) Enable execution of Unit tests

[Tommaso Teofili (JIRA)]

[ 
https://issues.apache.org/jira/browse/JOSHUA-253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15260253#comment-15260253
 ] 

Tommaso Teofili commented on JOSHUA-253:


as far as I can see the ant test target executes the test/run-all-tests.sh 
script, which looks for all the files that end in test-sh and executes them, so 
no Java unit test is executed as far as I know, but only the test.sh scripts in 
descendants of $JOSHUA/test directory.

> Enable execution of Unit tests
> --
>
> Key: JOSHUA-253
> URL: https://issues.apache.org/jira/browse/JOSHUA-253
> Project: Joshua
>  Issue Type: Test
>Affects Versions: 6.0
>Reporter: Lewis John McGibbney
> Fix For: 6.1
>
>
> As per our [discussion on this 
> topic|http://www.mail-archive.com/dev%40joshua.incubator.apache.org/msg00270.html],
>  [~teofili] correctly identified that unit level tests are not executed.
> We need to fix this such that they are.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (JOSHUA-253) Enable execution of Unit tests

[Lewis John McGibbney (JIRA)]

[ 
https://issues.apache.org/jira/browse/JOSHUA-253?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15260231#comment-15260231
 ] 

Lewis John McGibbney commented on JOSHUA-253:
-

[~teofili] [~post] where are the Unit tests we have to run? I've undertaken 
some analysis of the $JOSHUA_HOME/test diretory. As far as I can see they are 
invoked... so I am definitely missing something here. 

> Enable execution of Unit tests
> --
>
> Key: JOSHUA-253
> URL: https://issues.apache.org/jira/browse/JOSHUA-253
> Project: Joshua
>  Issue Type: Test
>Affects Versions: 6.0
>Reporter: Lewis John McGibbney
> Fix For: 6.1
>
>
> As per our [discussion on this 
> topic|http://www.mail-archive.com/dev%40joshua.incubator.apache.org/msg00270.html],
>  [~teofili] correctly identified that unit level tests are not executed.
> We need to fix this such that they are.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-joshua pull request: Fixed files in wrong folder, append...

[KellenSunderland]

Github user KellenSunderland commented on the pull request:

https://github.com/apache/incubator-joshua/pull/2#issuecomment-215097582
  
Thanks for the merge Matt.  I'll take a look at the commit you mention and 
rebase the next PR on top of it.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (JOSHUA-255) License headers for all bash scripts

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/JOSHUA-255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15260221#comment-15260221
 ] 

ASF GitHub Bot commented on JOSHUA-255:
---

GitHub user lewismc opened a pull request:

https://github.com/apache/incubator-joshua/pull/4

JOSHUA-255 License headers for all bash scripts

This issue addresses https://issues.apache.org/jira/browse/JOSHUA-255

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/lewismc/incubator-joshua JOSHUA-255

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-joshua/pull/4.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4


commit 3f55fe9e719ed578f417801c7abbeaca48e79e9d
Author: Lewis John McGibbney 
Date:   2016-04-27T14:15:13Z

JOSHUA-255 License headers for all bash scripts




> License headers for all bash scripts
> 
>
> Key: JOSHUA-255
> URL: https://issues.apache.org/jira/browse/JOSHUA-255
> Project: Joshua
>  Issue Type: Task
>Affects Versions: 6.0.5
>Reporter: Lewis John McGibbney
>Assignee: Lewis John McGibbney
> Fix For: 6.1
>
>
> Add license headers to all bash scripts in master branch



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] incubator-joshua pull request: JOSHUA-255 License headers for all ...

[lewismc]

GitHub user lewismc opened a pull request:

https://github.com/apache/incubator-joshua/pull/4

JOSHUA-255 License headers for all bash scripts

This issue addresses https://issues.apache.org/jira/browse/JOSHUA-255

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/lewismc/incubator-joshua JOSHUA-255

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/incubator-joshua/pull/4.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4


commit 3f55fe9e719ed578f417801c7abbeaca48e79e9d
Author: Lewis John McGibbney 
Date:   2016-04-27T14:15:13Z

JOSHUA-255 License headers for all bash scripts




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---