Bug 4545 - WS GRAM JSDL Support in GT 4.2
: WS GRAM JSDL Support in GT 4.2
Status: RESOLVED LATER
: GRAM
Campaign
: development
: Macintosh All
: P3 normal
: 4.4
Assigned To:
:
:
:
: 4044
  Show dependency treegraph
 
Reported: 2006-06-22 16:34 by
Modified: 2008-05-06 16:33 (History)


Attachments
GRAM JDD to JSDL mappings (5.35 KB, text/plain)
2006-07-05 16:50, Peter Lane
Details
GRAM JDD to JSDL mappings (6.41 KB, text/plain)
2006-07-07 13:19, Peter Lane
Details
GRAM JDD to JSDL mappings (6.43 KB, text/plain)
2006-07-27 19:06, Peter Lane
Details
XSLT jsdl to xrsl transformation, only core jsdl elements without any extensions (12.18 KB, application/octet-stream)
2006-11-20 03:42, Dawid Szejnfeld
Details
JSDL Delegation Pseudoschema (6.05 KB, text/plain)
2006-12-18 10:45, Peter Lane
Details


Note

You need to log in before you can comment on or make changes to this bug.


Description From 2006-06-22 16:34:03
Title: WS-GRAM JSDL Support in GT 4.2

Communities:

- UK NGS
- NAREGI
- (any others?)

Technologies:

Globus Resource Allocation Manager (GRAM)

Definition:

JSDL is an accepted GGF standard specification for describing grid jobs.  Since
JSDL became a standard, many groups have started to use it.  To accommodate
this emerging standard, we propose to extend our factory web service interface
with an operation that takes a JSDL document, while leaving the original
createManagedJob operation the same.  This will allow users to better plan when
they take on the transition from the current WS GRAM job description schema to
JSDL.

Then general plan is to add new operations and resource properties required to
support JSDL to the current WS GRAM service.  Resulting in a single WS GRAM
service that is capable of processing a job specified using the current JDD
schema as well as a job specified using the JSDL schema.  Clients written
before these JSDL enhancements will be able to interoperate with the new WS
GRAM service that supports both the current job description schema and JSDL. 
The gram clients: globusrun-ws and gramJob java API will be updated to identify
the job as either JDD or JSDL and use the matching WS GRAM service operations. 
The globusrun-ws job wrapper scripts (globus-job-*-ws) will not be altered and
will continue to specify job's in JDD format.

Deliverables:

1) Auto-generated classes for the JSDL and JSDL Posix extension schemas.
2) A document describing exactly how we propose to map JSDL to
JobDescriptionType and MultiJobDescriptionType.
3) A new createManagedJobFromJSDL operation in the ManagedJobFactoryPortType
WSDL that takes a JSDL document instead of a JobDescriptionType or
MultiJobDescriptionType document, converts the JSDL data structure to the
associated data structures for the latter two types, and calls the original
createManagedJob operation.
4) Updated ManagedJobPortType WSDL that includes a "JSDL" resource property
which contains the JSDL document.
5) A Java class named JsdlGramJob that extends GramJob but also takes JSDL
documents.
6) An updated globusrun-ws client that accepts JSDL in addition to our
proprietary job description format.
7) Additional unit tests for the new operation.
8) Additional scheduler tests that use JSDL instead of our prioprietary job
description format.
9) Documentation describing the new operation, the new resource property, the
mappings from #2, and client examples.


Tasks:

1) Create a document that describes all mappings and extensions needed to
convert a JSDL document into a JobDescriptionType or MultiJobDescriptionType
document.
2) Import the JSDL and JSDL Posix extension schemas into the 4.2 schema CVS
module.
3) Add a "createManagedJobFromJSDL" operation in the 4.2 factory schema.
4) Add a "JSDL" resource property to the 4.2 job schema.
5) Update Ant bulid files to generate Java classes for the new factory, job,
and JSDL schemas.
6) Update the WS-GRAM C bindings package to generate C code for the new
factory, job, and JSDL schemas.
7) Create a new JdslGramJob class that inherits from GramJob with additional
methods and functionality to handle JSDL documents.
8) Update globusrun-ws to detect which job description format is being used,
modify the document as needed, and call the appropriate job creation operation.
9) Add unit tests that stress the createManagedJobFromJSDL operation.
10) Add a unit test for the new "JSDL" resource property.
11) Add duplicate scheduler tests that use equivalent JSDL documents.
12) Add factory public interface documentation describing the
createManagedJobFromJSDL operation.
13) Add job public interface documentation describing the "JSDL" resource
property.
14) Add public interface documentation describing the mappings form JSDL to our
prioprietary job description format, including any extensions to JDSL that were
necessary.
------- Comment #1 From 2006-07-05 09:39:44 -------
Tasks 2, 3, and 5 are essentially done. I'm still debugging stub generation
errors, though. I've added JSDL extension schemas for GRAM and RFT, but have
yet to create the document required for task 1.
------- Comment #2 From 2006-07-05 11:54:58 -------
Actually, #3 is not done. I added the method skeleton but have not implemented
it yet.
------- Comment #3 From 2006-07-05 16:50:57 -------
Created an attachment (id=994) [details]
GRAM JDD to JSDL mappings

Deliverable #2
------- Comment #4 From 2006-07-05 17:13:43 -------
I'm changing task/deliverable #4 so that that resource property is simply a
reference to jsdl:JobDefinition instead of gram-job:JSDL (of type
jsdl:JobDefinition_Type). JSDL afterall is technically the schema rather than a
document that conforms to the schema. It also makes it more clear what you're
getting back.

Along these lines, I'm also changing the name of the new operation (deliverable
#3) to CreateManagedJobFromJSDLJobDefinition.
------- Comment #5 From 2006-07-07 13:19:09 -------
Created an attachment (id=997) [details]
GRAM JDD to JSDL mappings

Update element mapping document that makes things a little clearer and includes
changes based on development experience.
------- Comment #6 From 2006-07-11 13:15:32 -------
After much C WS Core debugging help from Joe, task #6 has been completed. I'm
almost done with the JsdlGramJob class (task #7/deliverable #5) so that I can
at least test the service to some extent.
------- Comment #7 From 2006-07-12 12:20:04 -------
Task #7 (deliverable #5) is done. I've been able to submit the following simple
JSDL job using the JsdlGramJob class:

<?xml version="1.0" encoding="UTF-8"?>
<jsdl:JobDefinition xmlns:jsdl="http://schemas.ggf.org/jsdl/2005/11/jsdl">
    <jsdl:JobDescription>
        <jsdl:Application>
            <jsdl-posix:POSIXApplication
xmlns:jsdl-posix="http://schemas.ggf.org/jsdl/2005/11/jsdl-posix">
                <jsdl-posix:Executable>/bin/date</jsdl-posix:Executable>
            </jsdl-posix:POSIXApplication>
        </jsdl:Application>
    </jsdl:JobDescription>
</jsdl:JobDefinition>
------- Comment #8 From 2006-07-27 19:06:47 -------
Created an attachment (id=1017) [details]
GRAM JDD to JSDL mappings
------- Comment #9 From 2006-07-27 19:09:58 -------
I updated globusrun-ws, but I'm having problems debugging seg faults. The JSDL
extension schemas might change again, so I may update for those changes first
before either getting help from Joe or handing it off to him altogether.
------- Comment #10 From 2006-10-04 10:20:49 -------
A status update is long overdue...

I was on vacation from the end of July through the first two weeks of August.

I did some work on rearranging the JSDL extensions and some other
implementation details after some feedback for most of the rest of August. I
also had a Chicago trip that prevented me from doing anything for almost a
week.

I was again off for the first week in September. I was at
GlobusWorld/OGF/GridWorld for the second week in September. The third week of
September I was occupied with preparations for another meeting in the fourth
week of September. This means that I got nothing done wrt this campaign in
September.

I've been playing catch up and doing some GRAM planning for the last few days.
There are some serious issues that are blocking MPICH-G development that I need
to attend to. Once I'm done with that I will hopefully be able to once again do
some work on this campaign.
------- Comment #11 From 2006-11-20 03:42:42 -------
Created an attachment (id=1128) [details]
XSLT jsdl to xrsl transformation, only core jsdl elements without any
extensions

We are preparing a plugin for gt4 for our stuff at PSNC in Poland. We have web
application which generates JSDL job descriptions so we need such thing. I
managed to write some xslt transformation from JSDL v1.0 to GT4 job description
xml. I used your mappings. I decided to translate only core elements without
adding any gt4 extension for jsdl but I think it could be helpful for you
anyway.
------- Comment #12 From 2006-12-01 17:19:31 -------
Now that SC is out of the way I can finish this stuff up. I had to help out
with getting some patches to VDT too. Not to mention some bug work.

Anyway, I abused the gram_jsdl_branch to work on the BES demo since that's
where all the JSDL work was and I had little time to mess with branching. That
means I need to clean some stuff up before merging to HEAD. I should be able to
finish that on Monday.

To avoid dealing with backwards compatibility issues, I changed the service to
accept the JSDL document in the extensions section instead of a replacement
create operation which is really a hack to get around the inability to have an
operation with the same local name but different namespace. All JDD elements
are optional, so the clients can simply wrap the JSDL document in an extensions
element inside of a jobDescription element and reuse the same submission code.
------- Comment #13 From 2006-12-07 19:30:05 -------
Things are working pretty good wrapped in the <extensions> element. I'm almost
done adding equivalent scheduler tests for JSDL. I'm considering disabling
automatic delegation in globusrun-ws for JSDL documents for lack of time. It
does work with manual delegation, though.
------- Comment #14 From 2006-12-15 19:49:03 -------
I finally fixed things for the local ouput, non-multi scheduler tests that I
added . The things left to address are automatic delegation with globusrun-ws,
multijob functionality, and documentation.
------- Comment #15 From 2006-12-18 10:45:36 -------
Created an attachment (id=1145) [details]
JSDL Delegation Pseudoschema

I'm attaching a pseudoschema that illustrates with incomplete examples the
differences between how the GRAM JDL and JSDL (via extensions) supports
delegated credential references.
------- Comment #16 From 2006-12-21 19:44:08 -------
We've decided to go ahead and create an extra service deployment with a WSDL
that overrides the current createManagedJob operation. I have the service work
done. 

I've been working on a new client API to abstract out all the different
versions of the service that we're supporting (currently 4.0, 4.2/JSDL, and
BES) since GramJob is an utter nightmare now. I've finished coding on the
client API but still need to make it compile and test everything.

I'm told Joe Bester will update globusrun-ws once I'm done with the Java work.
That means that multijob support and documentation will be outstanding.
Documentation is more important at this point, but if I have time in January
after writing the documentation I will try to get that working.
------- Comment #17 From 2008-05-06 16:33:12 -------
I'm closing this campaign.  It can reopen or rewrite a new campaign when we are
ready to take on jsdl.