Bug 4412 - Grid Service Audit and Accounting Query Interface
: Grid Service Audit and Accounting Query Interface
Status: RESOLVED FIXED
: GRAM
Campaign
: development
: Macintosh All
: P3 normal
: TG
Assigned To:
:
: Auditing
:
: 4409
  Show dependency treegraph
 
Reported: 2006-05-17 11:38 by
Modified: 2007-02-14 15:27 (History)


Attachments
OGSA-DAI TeraGrid Resoruces (40.30 KB, application/x-gzip)
2006-06-12 16:11, Peter Lane
Details
Modified OD TG Resources (103.06 KB, application/x-gzip)
2006-06-14 13:24, Peter Lane
Details


Note

You need to log in before you can comment on or make changes to this bug.


Description From 2006-05-17 11:38:56
Title: Grid Service Audit and Accounting Query Interface

Technologies:    OSGA-DAI, Globus Resource Allocation Manager (GRAM)

Definition:

TeraGrid wants to enable gateways (grid clients/users) to retrieve usage
information for a GRAM job.  Ideally, the grid user would not have to learn the
details of which local resource manager was used, nor the details of the local
accounting system.  Grid services return a unique handle/EPR/contact/... from a
service request.  This handle is used to control the WS resource (i.e. to
cancel a job).  This handle could also be converted into a Grid job ID used to
fetch information related to the job, like usage info, the local RM job id, the
Grid ID for the service used to stage a file, ...

It would be ideal for a grid client to interact with a grid service for access
to this audit and accounting information.  The client should not have to learn
the details of where this information is stored.  For example, the gram audit
DB is on host X in a PostgresSQL DB, the job accounting DB is on host Y in a
MySQL DB.  Also, the client should not have to learn the relationship between
the grid provider's DBs.  For example, on TeraGrid, to identify a unique job
usage record requires the local job id, the compute resource name (a human
determined label) and a approximate time of when the job was submitting (in
order to handle repeated or rolling job ids).

For the simple use case of - client provides a Grid Job ID / service returns
the usage amount, an application-specific service could be written to provide
this capability.  However, we anticipate that more open ended SQL querying of
audit and accounting records will be desired.  If so, then it would make sense
to provide this interface through the OGSA-DAI port types.  Essentially the
client will have to call the OGSA-DAI 'perform' operation with an XML document
that specifies the activities and parameters he wishes to execute.  This will
look something like:

<?xml version="1.0" encoding="UTF-8"?>
<perform xmlns="http://ogsadai.org.uk/namespaces/2005/10/types">
  <session/>
  <getChargeFromGridJobId name="myActivity">
    <GridJobId>theGridJobId</GridJobId>
    <chargeOutput name="charge"/>
  </getChargeFromGridJobId >
</perform>

The client will get a response document which contains the result.  For
example:

<?xml version="1.0" encoding="UTF-8"?>
<response xmlns="http://ogsadai.org.uk/namespaces/2005/10/types">
  <session id="sesssionId"/>
  <request status="COMPLETED"/>
  <result name="myActivity" status="COMPLETED"/>
  <result name="charge" status="COMPLETED"/>
     11.56
  </result>
</response>

Using the OGSA-DIA Java client toolkit, a client/gateway would code something
like:

// Set up stuff required once
String handle =
    "http://localhost:8080/wsrf/services/ogsadai/DataService";
String id = "MySQLResource";
DataService service =
    WSRFServiceFetcher.getInstance().getDataService(handle, id);

// Code required per query
getChargeFromGridJobId usageQuery = new
    getChargeFromGridJobId(gjid);
service.perform(usageQuery);
double charge = usageQuery.getCharge();


Deliverables:

1) General OGSA DAI Software Version
2) Specific getCharge Activity software
3) Simple client getCharge prototype

Tasks:

1) install OGSA-DAI WSRF 2.2 - Download from:
http://www.ogsadai.org.uk/downloads/
2) Create a data service called DataService
3) Create a getCharge "activity" that will connect to the GRAM Audit DB and the
TG accounting DB to fetch the info
4) Expose this resource to the data service
5) write a simple client to call the OGSA-DAI 'perform' operation with a
getCharge
6) Test

Time Estimate:

5 days
------- Comment #1 From 2006-06-12 16:11:09 -------
Created an attachment (id=979) [details]
OGSA-DAI TeraGrid Resoruces

Ally completed tasks 1-5 and I've been able to test this (task 6) using local
services and DBs. I'm attaching the tarball that Ally gave me that contains the
TG-specific resources for OGSA-DAI.
------- Comment #2 From 2006-06-14 13:24:46 -------
Created an attachment (id=982) [details]
Modified OD TG Resources

I'm attaching a modified version that I just hacked to allow any number of
audit databases. I convinced Stu that what we want is one central OGSA-DAI
deployment that has one resource for the central accounting database and a
resource for each audit database that exists (we're assuming one per GRAM
deployment). This updated version will resolve the resource ID (obtained in
turn from the JGID host name) to a connection name. This connection name will
then be used to fetch both the appropriate DB query as well as the DB
connection data used to contact the desired audit database.