I trust you mean "save you" not "prevent you"... :)
In any case, what we'd need here, if we're going to avoid commandline
hacking with `crontab -e`, is that a process can be run by root or
hudsonBuild as someone's logged-in user so as to pick up that user's
LDAP credentials and apply that to its ability to write files on
dev.eclipse.org. Basically we need hudson to be able to `su - someuser`.
So, here's the scenario:
I log into Hudson as nickb; I push a button (provided by a to-be-built
Hudson plugin) while logged in to publish some given build, which can
therefore do a cp or scp or rsync as ni...@build.eclipse.org to push
files locally within the nfs mounts, or to ni...@[somewhere else] if the
right ssh keys are in place. (eg., to Sourceforge? Apache Maven repo?)
Restrictions need to be in place such that I, nickb, can't publish a
build as dacarver and circumvent the filesystem permissions using some
Hudson-based override (ie., because I'm in callisto-dev or hudson-admin
or some "special" user group).
Plugin would only need to provide simple UI and form a linkage between
LDAP login on http://build.eclipse.org/hudson/ and allowing sudo access
to u...@build.eclipse.org on the filesystem. Individual 'promote this
build!' buttons would need at least one or two properties to define HOW
and WHERE to promote. So, for simplicity, why not have the job define
${projRelengDir}/promote.xml and use the accompanying
${projRelengDir}/promote-${buildType}.properties (if avail) else
${projRelengDir}/promote.properties; that way there's no need to
configure it - it'll just assume your file is in a well-defined path,
and if you push the button and the file doesn't exist, you'll simply get
an error message explaining where the file should be, and a link to the
wiki explaiing how to create it.
---
Alternatively, the lock file workaround is:
a) hudsonBuild writes a file
b) user sees the file and responds (via crontab watcher (running as
u...@build.eclipse.org) or any http get method you'd prefer, like an
ECF-based eclipse plugin that can use RSE to publish the files over
ssh). If you're a KDE fan you could use a kioslave to copy from
fish://u...@build.eclipse.org to fish://u...@dev.eclipse.org.
---
The first scenario is IMHO cleaner than the second scenario and would
reuse the existing LDAP access controls & Hudson UI, without making
people "slum it" via commandline. :)
Denis, is such a thing possible? Is there API documentation for the
single-signon stuff for the Eclipse sites, so it can be used by a Hudson
plugin?
N
On 01/31/2010 10:48 AM, Chris Aniszczyk wrote:
As an outsider, anything that would prevent me from hacking cron jobs
in crontab to publish builds would be appreciated.
On Sun, Jan 31, 2010 at 9:26 AM, David Carver<d_a_car...@yahoo.com> wrote:
Denis is there a place where the security requirements are documented for
moving files between systems at eclipse? The reason I'm asking is that it
would help me come up with an integrated solution (i.e. possibly updating
the SCP plugin for specific user logins). Let me see what type of license
the SCP plugin at Hudson is under. I can start with that, as a basis for a
possible DASH project for Hudson specific plugins and see what happens from
there. I'm fine for the Cron job idea, as long as it's documented.
Dave
Nick Boldt wrote:
Dude, "creating a cron job" is like 5 mins of work. Copy, paste, issue
command `crontab -e`, set polling time/interval, and vavoom, done. Hardly a
high barrier. Getting yourself a bash account on build.eclipse.org
<http://build.eclipse.org> takes longer (and that only requires filing a bug
and waiting). :)
As to a Hudson Plugin project in Dash, sure... cuz it's that or at SF or
Google Code or wherever else code is hosted. If you want to jump thru the
hoops to make it happen, by all means do.
As to slaves, one workaround there is to enable the 'publishing plugin'
which allows the output of a build to be pushed into another Hudson instance
(then poll that instance for updates); alternatively, if you tie your job to
a single slave, it's just as effective as being tied to master: poll the
slave/master on which your build runs.
At some point, when we have the option to run across multiple
slaves/platforms, the only requirements will therefore be that the crontab
script poll all the appropriate places and that the slaves all support
ssh/scp/rsync in order to be pollable. Or failing that, ftp/http, I suppose.
N
On Fri, Jan 29, 2010 at 10:51 AM, David Carver<d_a_car...@yahoo.com
<mailto:d_a_car...@yahoo.com>> wrote:
Well, the SCP plugin is Open Source, so I'm sure if some
enterprising committer wanted to modify it to allow more
configuration options one could do so. These types of concerns
are why I think Dash may need to have a Hudson Plugin subproject.
This way you can address the eclispe specific environment concerns.
Nick, how does your Batch Task/Cron idea work in a distributed
system when Slave machines? What about slave machines that
aren't Linux based? How does it work with Windows based
machines? Some things to keep in mind.
I just think having to make users create cron jobs is just another
barrier that isn't necessarily necessary. I'd still prefer
something that was more integrated instead of doing batch magic
which can be still as insecure as SCP rights. Just takes one
wrong batch job executing at the wrong access level to mess things up.
Dave
On 01/29/2010 06:24 AM, Denis Roy wrote:
+1
Cron jobs that poll are not nearly as elegant, but they do allow
for better security.
D.
On 01/28/2010 10:59 PM, Nick Boldt wrote:
Problem with SCP plugin is that it uses global scp access rights
to move files; it doesn't seem to be able to key in to the LDAP
database so that I can push files to
ni...@dev.eclipse.org:~/downloads/
<mailto:ni...@dev.eclipse.org:%7E/downloads/>, for example.
My workaround is to have Hudson write to its own workspace (eg.,
create a .lock file that says "job foo's build #34 is ready to
promote") and then have nickb's crontab look for that file every
minute or two; if found, it can perform the publish and either
delete the .lock file, or record somehow that it's already
pushed that build so as to not do so again... unless the Batch
Task is kicked again to refresh the file.
So, like with the signing genie, we'd have a process that can
pass information between us...@dev.eclipse
<mailto:us...@dev.eclipse> and the shared user hudsonBuild in
order to allow uease of use AND security.
N
On 01/28/2010 05:02 PM, David Carver wrote:
If we are concerned about giving Hudson write access to the
downloads
directory, then we might want to consider the Batch Tasks
Plugin that
Nick has suggested. You could run an ftp/scp command from there
and ftp
the files over to the download server. At least this way it is
running
under specific committer ids, which would then limit it to
specific
directories instead of the whole thing.
There is also the SCP plugin for Hudson which allows for a secure
connection and copying of files.
http://wiki.hudson-ci.org/display/HUDSON/SCP+plugin
Dave
On 01/28/2010 12:41 PM, Kim Moir wrote:
It would be nice to be able to tag cvs projects as part of a
hudson
build. Today, when we run our integration builds, we tag our
builder
and our map file project so that the build is reproducible.
Once we
have test hardware at the foundation and can run our builds on
the
hudson install on build.eclipse.org
<http://build.eclipse.org>, the only way to do this is to
have cronjob entries that correspond the time our build
starts, which
isn't 100% foolproof. That being said, I totally understand
Denis'
concern with the Hudson user having write access to the larger
download and source filesystem. Not good from a security
perspective.
Kim
*David Carver<d_a_car...@yahoo.com>
<mailto:d_a_car...@yahoo.com>*
Sent by: dash-dev-boun...@eclipse.org
<mailto:dash-dev-boun...@eclipse.org>
01/28/2010 11:54 AM
Please respond to
Tools for Committer Community<dash-dev@eclipse.org>
<mailto:dash-dev@eclipse.org>
To
Tools for Committer Community<dash-dev@eclipse.org>
<mailto:dash-dev@eclipse.org>
cc
Subject
Re: [dash-dev] Streamlining the Athena promote process
(was Re:
[gmf-dev] Head up: last night's build failure was a real
failure in
org.eclipse.gmf.xpand.migration)
Promotion should only require write access to to the various
download
area folders for the projects.
It wouldn't necessarily have to have tagging priviledges to
CVS/SVN/Git as it stands right now all projects use MAP files
for this.
Dave
On 01/28/2010 08:24 AM, Denis Roy wrote:
For Hudson to do the promotion itself, I assume the
hudsonBuild user
would need write access to most, if not all of the downloads
area. If
the promotion process involves tagging or committing to CVS, then
hudsonBuild would need commit access to CVS.
Ick.
On 01/28/2010 11:11 AM, David Carver wrote:
Nick, Hudson already has plugins that would allow the
promotion. The
problem is that the user that Hudson runs under does not have
rights
to the download server to be able to push the necessary bits.
We have
the necessary plugins installed in Hudson, just don't have the
access
rights.
Instead of jumping through work arounds and hoops we really
need to
address the problem at hand. Hudson should have necessary access
rights to be able to do the promotion itself.
Dave
On 01/27/2010 11:43 PM, Nick Boldt wrote:
Theoretically, it should be posible to create a hudson job
that would
be used simply to promote the latest clean builds by flipping
a bit to
notify a listening process (namely a crontab script) that it's
time to
promote.
But does having a button on a webpage really make publishing
easier
than ssh'ing to a server and running a script? The same code
that the
crontab runs can be run on demand, and you could wrap that
line of
code w/ a wrapper script such that all you'd need to do is ssh to
build.eclipse and run "~/p" to push your bits. Only marginally
easier
than logging in to build.eclipse.org
<http://build.eclipse.org> to push a button.
Would an Eclipse plugin would be perhaps a better idea? Perhaps
something graphically modeled (hint: GMF? Zest?), with not just a
button but a view of your latest builds in Hudson and their
test results?
Or, perhaps a Hudson plugin is better, such that in addition
to the
build button we already have, we could also have a promote
button?
N
On 01/25/2010 09:54 AM, Anthony Hunter wrote:
Hi Nick,
With GMF and the common modeling build, have a web page with two
buttons, build and promote
With the Athena build, have a web page with a build button,
but no
promote yet. You last communicated promote must be manually
ran from the
command line or cron.
> From my point of view, the lack of a quick easy promote have
Athena a
hard sell.
To be honest however, I have not had any time to look further
that
Athena for GEF.
Once GEF and EMF have fully moved, the rest of the modeling
stack (and
GMF) should have no excuse to move.
And the goal should be / is for all modeling projects to share
the same
build technology.
Cheers...
Anthony
-- Anthony Hunter _mailto:antho...@ca.ibm.com_
Software Development Manager
IBM Rational Software: Aurora / Modeling Tools
Phone: 613-270-4613
Inactive hide details for Nick Boldt ---2010/01/25 12:54:34
AM---If only
there was a better way... like building against updateNick Boldt
---2010/01/25 12:54:34 AM---If only there was a better way...
like
building against update sites
From:
Nick Boldt _<nickbo...@gmail.com>
<mailto:nickbo...@gmail.com>_<mailto:nickbo...@gmail.com>
To:
"GMF Project developer discussions." _<gmf-...@eclipse.org>
<mailto:gmf-...@eclipse.org>_
<mailto:gmf-...@eclipse.org>
Date:
2010/01/25 12:54 AM
Subject:
Re: [gmf-dev] Head up: last night's build failure was a real
failure in
org.eclipse.gmf.xpand.migration
------------------------------------------------------------------------
If only there was a better way... like building against update
sites
instead of (or in addition to) SDK zips...
oh, wait... there is!
_
__http://wiki.eclipse.org/Common_Build_Infrastructure/Defining_Binary_Dependencies_
With Galileo SR2 nearly complete, does GMF plan to move to
Athena/Hudson during the Helios cycle? If not, what blocks
you? I'd
like a list of blocking requirements so I can better
prioritize Athena
TODOs and get it into a form that will meet more (if not all)
of your
needs.
BTW, the tagandrelease system works just as well for Athena
builds as
for Modeling builds; the only difference is that you have full
control
over it (ie., no CVS commit issues) and would run it on
build.eclipse.org<http://build.eclipse.org> instead of
modeling.eclipse.org<http://modeling.eclipse.org>.
Cheers,
Nick
On Wed, Jan 20, 2010 at 9:55 AM, Anthony Hunter
_<antho...@ca.ibm.com> <mailto:antho...@ca.ibm.com>_
<mailto:antho...@ca.ibm.com> wrote:
> Hi Team,
>
> Releng update. I am trying to restore
org.eclipse.releng.tools.tagandrelease
> for GMF. It runs from cron on modeling.eclipse.org
<http://modeling.eclipse.org>. It checks CVS
for new
> changes, tags and releases the new code in the gmf map files
and then
runs a
> build. This would be a completely hands off run a nightly
Integration
build
> when a change is committed.
>
> Everything is now working fine, but the dependencies
calculator does
not
> work when firing a new GMF build.
>
> This is he explanation for the constant failing GMF builds,
you need
to pick
> the correct dependencies as well as using the linux-gtk-x86_64
Eclipse SDK.
>
> This is
_https://bugs.eclipse.org/bugs/show_bug.cgi?id=299802_ that
we have
> yet to fix.
>
> Last night I ran a integration build manually selecting what
I think
was the
> latest dependencies:
> _
__http://modeling.eclipse.org/modeling/gmf/gmf/downloads/drops/2.3.0/I201001192155/buildlog.txt_
>
> It has failed in org.eclipse.gmf.xpand.migration
>
> Can the tooling confirm?
>
> The dependencies were:
> -URL
> _
__http://download.eclipse.org/eclipse/downloads/drops/I20100119-0800/eclipse-SDK-I20100119-0800-linux-gtk-x86_64.tar.gz_
> -URL
> _
__http://download.eclipse.org/modeling/emf/emf/downloads/drops/2.6.0/I201001101746/emf-xsd-SDK-I201001101746.zip_
> -URL
> _
__http://download.eclipse.org/modeling/mdt/uml2/downloads/drops/3.1.0/S200912141514/mdt-uml2-SDK-3.1.0M4.zip_
> -URL
> _
__http://download.eclipse.org/tools/orbit/downloads/drops/R20090825191606/orbitBundles-R20090825191606.map_
> -URL
> _
__http://modeling.eclipse.org/modeling/emf/query/downloads/drops/1.4.0/S200912161005/emf-query-SDK-1.4.0M4.zip_
> -URL
> _
__http://modeling.eclipse.org/modeling/emf/transaction/downloads/drops/1.4.0/S200912161108/emf-transaction-SDK-1.4.0M4.zip_
> -URL
> _
__http://modeling.eclipse.org/modeling/emf/validation/downloads/drops/1.4.0/S200912161043/emf-validation-SDK-1.4.0M4.zip_
> -URL
> _
__http://download.eclipse.org/tools/gef/downloads/drops/3.6.0/I201001151504/GEF-SDK-I201001151504.zip_
> -URL
> _
__http://download.eclipse.org/modeling/m2m/qvtoml/downloads/drops/3.0.0/S200912160721/m2m-qvtoml-SDK-3.0.0M4.zip_
> -URL
> _
__http://download.eclipse.org/modeling/mdt/ocl/downloads/drops/3.0.0/I201001070745/mdt-ocl-SDK-I201001070745.zip_
>
> Cheers...
> Anthony
> --
> Anthony Hunter _mailto:antho...@ca.ibm.com_
> Software Development Manager
> IBM Rational Software: Aurora / Modeling Tools
> Phone: 613-270-4613
>
> _______________________________________________
> gmf-dev mailing list
> _gmf-...@eclipse.org_<mailto:gmf-...@eclipse.org>
> _https://dev.eclipse.org/mailman/listinfo/gmf-dev_
>
>
-- Nick Boldt :: JBoss by Red Hat
Productization Lead :: JBoss Tools& Dev Studio
Release Engineer :: Dash Athena _
__http://nick.divbyzero.com_<http://nick.divbyzero.com/>
_______________________________________________
gmf-dev mailing list _
__gmf-...@eclipse.org_<mailto:gmf-...@eclipse.org> _
__https://dev.eclipse.org/mailman/listinfo/gmf-dev_
_______________________________________________
dash-dev mailing list
_dash-...@eclipse.org_<mailto:dash-dev@eclipse.org>
_https://dev.eclipse.org/mailman/listinfo/dash-dev_
_______________________________________________
dash-dev mailing list
dash-dev@eclipse.org<mailto:dash-dev@eclipse.org>
https://dev.eclipse.org/mailman/listinfo/dash-dev
_______________________________________________
dash-dev mailing list
dash-dev@eclipse.org<mailto:dash-dev@eclipse.org>
https://dev.eclipse.org/mailman/listinfo/dash-dev
_______________________________________________
dash-dev mailing list
dash-dev@eclipse.org<mailto:dash-dev@eclipse.org>
https://dev.eclipse.org/mailman/listinfo/dash-dev
-- EclipseCon 2010*Denis Roy*
Manager, IT Infrastructure
Eclipse Foundation, Inc. -- http://www.eclipse.org/
Office: 613.224.9461 x224 (Eastern time)
denis....@eclipse.org<mailto:denis....@eclipse.org>
_______________________________________________
dash-dev mailing list
dash-dev@eclipse.org<mailto:dash-dev@eclipse.org>
https://dev.eclipse.org/mailman/listinfo/dash-dev
_______________________________________________
dash-dev mailing list
dash-dev@eclipse.org<mailto:dash-dev@eclipse.org>
https://dev.eclipse.org/mailman/listinfo/dash-dev
--
Nick Boldt :: JBoss by Red Hat
Productization Lead :: JBoss Tools& Dev Studio
Release Engineer :: Dash Athena
http://nick.divbyzero.com
------------------------------------------------------------------------
_______________________________________________
dash-dev mailing list
dash-dev@eclipse.org
https://dev.eclipse.org/mailman/listinfo/dash-dev
_______________________________________________
dash-dev mailing list
dash-dev@eclipse.org
https://dev.eclipse.org/mailman/listinfo/dash-dev
--
Nick Boldt :: http://nick.divbyzero.com
Release Engineer :: Eclipse Modeling & Dash Athena
_______________________________________________
dash-dev mailing list
dash-dev@eclipse.org
https://dev.eclipse.org/mailman/listinfo/dash-dev
