http://git-wip-us.apache.org/repos/asf/oodt/blob/098cc4fa/product/src/site/xdoc/tutorials/lh/index.xml ---------------------------------------------------------------------- diff --git a/product/src/site/xdoc/tutorials/lh/index.xml b/product/src/site/xdoc/tutorials/lh/index.xml deleted file mode 100755 index cb3de01..0000000 --- a/product/src/site/xdoc/tutorials/lh/index.xml +++ /dev/null @@ -1,718 +0,0 @@ -<?xml version="1.0" encoding="UTF-8"?> -<!-- - Licensed to the Apache Software Foundation (ASF) under one or more - contributor license agreements. See the NOTICE file distributed with - this work for additional information regarding copyright ownership. - The ASF licenses this file to You under the Apache License, Version 2.0 - (the "License"); you may not use this file except in compliance with - the License. You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. ---> -<document> - <properties> - <title>Serving Large Products</title> - <author email="[email protected]">Sean Kelly</author> - </properties> - - <body> - <section name="Serving Large Products"> - <p>In the <a href="../qh/">last tutorial</a>, we created a query - handler and "installed" it in a product server. We could - query it for products (mathematical constants) using the - XMLQuery's postfix boolean stacks. The handler would return - results by embedding them in the returned XMLQuery. Now we'll - return larger products that live outside of the XMLQuery. - </p> - </section> - - <section name="What's Large?"> - <p>There's a <a - href="http://www.worldslargestthings.com/california/clam.htm";>giant - clam</a> at Pismo Beach, a <a - href="http://www.worldslargestthings.com/kansas/cawkercity.htm";>giant - ball of twine</a> in Kansas, and for those who drive SUVs, a - <a - href="http://www.worldslargestthings.com/missouri/gaspump.htm";>giant - gas pump</a>. For the OODT framework, large is similarly hard to define. - </p> - - <p>One of the original architects of the OODT framework thought - that putting a products result in with the query meant that - you'd never lose the separation between product and the query - that generated it. I'm not sure I see the value in that, but - regardless, it posed a practical challenge: an - <code>XMLQuery</code> object in memory with one or two large - results in it will exhaust the Java virtual machine's available - memory. - </p> - - <p>It's even worse in when the XMLQuery is expressed as a - textual XML document. In this case, a binary product must be - encoded in a text format (we use <a - href="ftp://ftp.rfc-editor.org/in-notes/rfc2045.txt";>Base64</a>), - making the XMLQuery in XML format even larger than as a Java - object. Moreover, those XML documents must be parsed at some - time to reconstitute them as Java objects. We use a DOM-based - parser, which holds the entire document in memory. Naturally, - things tend to explode at this rate. - </p> - - <p>There is a way out of the quagmire, though. Instead of - writing a <code>QueryHandler</code>, write a - <code>LargeProductQueryHandler</code>. A - <code>QueryHandler</code> puts <code>Result</code> objects - into the <code>XMLQuery</code> which hold the entire product. - A <code>LargeProductQueryHandler</code> puts - <code>LargeResult</code> objects which hold <em>a reference to - the product</em>. - </p> - </section> - - <section name="Large Handlers and Large Results"> - <p>The OODT framework provides an extension to the - <code>QueryHandler</code> interface called - <code>jpl.eda.product.LargeProductQueryHandler</code>. This - interface adds two methods that you must implement: - </p> - - <ul> - <li><code>retrieveChunk</code>. This method returns a byte - array representing a chunk of the product. The OODT - framework calls this method repeatedly to gather chunks of - the product for the product client. It takes a <em>product - ID</em> (a string) that identifies which product is being - retrieved. It also takes an byte offset into the product - data and a size of the byte chunk to return. You return the - matching chunk. - </li> - - <li><code>close</code>. This method is called by the OODT - framework to tell the query handler it's done getting a - product. It takes a <em>product ID</em> that tells which - product is no longer being retrieved. You use this method - to perform any cleanup necessary. - </li> - </ul> - - <p>Because it extends the <code>QueryHandler</code> interface, - you still have to implement the <code>query</code> method. - However, as a <code>LargeProductQueryHandler</code>, you can - add <code>LargeResult</code> objects to the - <code>XMLQuery</code> passed in. <code>LargeResult</code>s - identify the <em>product ID</em> (string) that the OODT - framework will later use when it calls - <code>retrieveChunk</code> and <code>close</code>. - </p> - - <p>For example, suppose you're serving large images by - generating them from various other data sources: - </p> - - <ol> - <li>The <code>query</code> method would examine the user's - query, consult the various data sources, and generate the - image, storing it in a temporary file. It would also assign - a string <em>product ID</em> to this file, use that product - ID in a <code>LargeResult</code> object, add the - <code>LargeResult</code> to the <code>XMLQuery</code>, and - return the modified <code>XMLQuery</code>. - </li> - - <li>Shortly afterward, the OODT framework will repeatedly call - the <code>retrieveChunk</code> method. This method would - check the <em>product ID</em> passed in and locate the - corresponding temporary file generated earlier by the - <code>query</code> method. It would index into the file by - the offset requested by the framework, read the number of - bytes requested by the framework, package that up into a - byte array, and return it. Eventually, the OODT framework - will have read the entire product this way. - </li> - - <li>Lastly, the OODT framework will call the - <code>close</code> method. This method would check the - <em>product ID</em> and locate and delete the temporary - file. - </li> - </ol> - - <p>To put this into practice, let's create a - <code>LargeProductQueryHandler</code> that serves files out of - the product server's filesystem. - </p> - </section> - - <section name="Writing the Handler"> - <p>We'll develop a <code>FileHandler</code> that will serve - files out of the product server's filesystem. Providing - filesystem access through the OODT framework in this way is - probably not a very good idea (after all, product clients - could request copies of sensitive files), but for a - demonstration it'll do. - </p> - - <p>Because files can be quite large, we'll use a - <code>LargeProductQueryHandler</code>. It will serve queries - of the form - </p> - - <p><code>file = <var>path</var></code></p> - - <p>where <var>path</var> is the full path of the file the user - wants. The handler will add <code>LargeResult</code>s to the - XMLQuery, and the <em>product ID</em> will just simply be the - <var>path</var> of the requested file. The - <code>retrieveChunk</code> method will open the file with the - given product ID (which is just the path to the file) and - return a block of data out of it. The <code>close</code> - method won't need to do anything, since we're not creating - temporary files or making network conncetions or anything; - there's just nothing to clean up. - </p> - - <subsection name="Getting the Path"> - <p>First, let's create a utility method that takes the - <code>XMLQuery</code> and returns a <code>java.io.File</code> - that matches the requested file. Because the query takes the form - </p> - - <p><code>file = <var>path</var></code></p> - - <p>there should be three <code>QueryElement</code>s on the "where" stack:</p> - - <ol> - <li>The zeroth (topmost) has role = <code>elemName</code> - and value = <code>file</code>. - </li> - <li>The first (middle) has role = <code>LITERAL</code> and - value = the <var>path</var> of the file the user wants. - </li> - <li>The last (bottom) has role = <code>RELOP</code> and - value = <code>EQ</code>. - </li> - </ol> - - <p>We'll reject any other query by returning <code>null</code> - from this method. Further, if the file named by the - <var>path</var> doesn't exist, or if it's not a file (for - example, it's a directory or a socket), we'll return <code>null</code>. - </p> - - <p>Here's the start of our <code>FileHandler.java</code>:</p> - - <source>import java.io.File; -import java.util.List; -import jpl.eda.product.LargeProductQueryHandler; -import jpl.eda.xmlquery.QueryElement; -import jpl.eda.xmlquery.XMLQuery; -public class FileHandler - implements LargeProductQueryHandler { - private static File getFile(XMLQuery q) { - List stack = q.getWhereElementSet(); - if (stack.size() != 3) return null; - QueryElement e = (QueryElement) stack.get(0); - if (!"elemName".equals(e.getRole()) - || !"file".equals(e.getValue())) - return null; - e = (QueryElement) stack.get(2); - if (!"RELOP".equals(e.getRole()) - || !"EQ".equals(e.getValue())) - return null; - e = (QueryElement) stack.get(1); - if (!"LITERAL".equals(e.getRole())) - return null; - File file = new File(e.getValue()); - if (!file.isFile()) return null; - return file; - } -}</source> - </subsection> - <subsection name="Checking the MIME Type"> - <p>Recall that the user can say what MIME types of products - are acceptable by specifying the preference list in the - XMLQuery. This lets a product server that serves, say, - video clips, convert them to <code>video/mpeg</code> - (MPEG-2), <code>video/mpeg4-generic</code> (MPEG-4), - <code>video/quicktime</code> (Apple Quicktime), or some - other format, in order to better serve its clients. - </p> - - <p>Since our product server just serves <em>files of any - format</em>, we won't really bother with the list of - acceptable MIME types. After all, the - <code>/etc/passwd</code> file <em>could</em> be a JPEG - image on some systems. (Yes, we could go through the - extra step of determining the MIME type of a file by - looking at its extension or its contents, but this is an - OODT tutorial, not a something-else-tutorial!) - </p> - - <p>However, we will honor the user's wishes by labeling the - result's MIME type based on what the user specifies in the - acceptable MIME type list. So, if the product client says - that <code>image/jpeg</code> is acceptable and the file is - <code>/etc/passwd</code>, we'll call - <code>/etc/passwd</code> a JPEG image. However, we won't - try to read the client's mind: if the user wants - <code>image/*</code>, then we'll just say it's a binary - file, <code>application/octet-stream</code>. - </p> - - <p>Here's the code:</p> - - <source>import java.util.Iterator; -... -public class FileHandler - implements LargeProductQueryHandler { - ... - private static String getMimeType(XMLQuery q) { - for (Iterator i = q.getMimeAccept().iterator(); - i.hasNext();) { - String t = (String) i.next(); - if (t.indexOf('*') == -1) return t; - } - return "application/octet-stream"; - } -}</source> - </subsection> - - <subsection name="Inserting the Result"> - <p>Once we've got the file that the user wants and the MIME - type to call it, all we have to do is insert the - <code>LargeResult</code>. Remember that it's the - <code>LargeResult</code> that tells the OODT framework what - the <em>product ID</em> is for later - <code>retrieveChunk</code> and <code>close</code> calls. - The <em>product ID</em> is passed as the first argument to - the <code>LargeResult</code> constructor. - </p> - - <p>We'll write a utility method to insert the <code>LargeResult</code>:</p> - - <source>import java.io.IOException; -import java.util.Collections; -import jpl.eda.xmlquery.LargeResult; -... -public class FileHandler - implements LargeProductQueryHandler { - ... - private static void insert(File file, String type, - XMLQuery q) throws IOException { - String id = file.getCanonicalPath(); - long size = file.length(); - LargeResult lr = new LargeResult(id, type, - /*profileID*/null, /*resourceID*/null, - /*headers*/Collections.EMPTY_LIST, size); - q.getResults().add(lr); - } -}</source> - - </subsection> - - <subsection name='Handling the Query'> - <p>With our three utility methods in hand, writing the - required <code>query</code> method is a piece of cake. Here - it is: - </p> - - <source>import jpl.eda.product.ProductException; -... -public class FileHandler - implements LargeProductQueryHandler { - ... - public XMLQuery query(XMLQuery q) - throws ProductException { - try { - File file = getFile(q); - if (file == null) return q; - String type = getMimeType(q); - insert(file, type, q); - return q; - } catch (IOException ex) { - throw new ProductException(ex); - } - } -}</source> - - <p>The <code>query</code> method as defined by the - <code>QueryHandler</code> interface (and extended into the - <code>LargeProductQueryHandler</code> interface) is allowed - to throw only one kind of checked exception: - <code>ProductException</code>. So, in case the - <code>insert</code> method throws an - <code>IOException</code>, we transform it into a - <code>ProductException</code>. - </p> - - <p>Now there are just two more required methods to implement, - <code>retrieveChunk</code> and <code>close</code>. - </p> - </subsection> - - <subsection name='Blowing Chunks'> - <p>The OODT framework repeatedly calls handler's - <code>retrieveChunk</code> method to get chunks of the - product, evenutally getting the entire product (unless the - product client decides to abort the transfer). For our file - handler, retrieve chunk just has to - </p> - <ol> - <li>Make sure the file specified by the <em>product ID</em> - still exists (after all, it could be deleted at any time, - even before the first <code>retrieveChunk</code> got - called). - </li> - <li>Open the file.</li> - <li>Skip into the file by the requested offset.</li> - <li>Read the requested number of bytes out of the file.</li> - <li>Return those bytes.</li> - <li>Close the file.</li> - </ol> - - <p>We'll write a quick little <code>skip</code> method to skip - into a file's input stream: - </p> - - <source>private static void skip(long offset, - InputStream in) throws IOException { - while (offset > 0) - offset -= in.skip(offset); -}</source> - - <p>And here's another little utility method to read a - specified number of bytes out of a file's input stream: - </p> - - <source>private static byte[] read(int length, - InputStream in) throws IOException { - byte[] buf = new byte[length]; - int numRead; - int index = 0; - int toRead = length; - while (toRead > 0) { - numRead = in.read(buf, index, toRead); - index += numRead; - toRead -= numRead; - } - return buf; -}</source> - - <p>(By now, you're probably wondering why we just didn't use - <code>java.io.RandomAccessFile</code>; I'm wondering that - too!)</p> - - <p>Finally, we can implement the required - <code>retrieveChunk</code> method: - </p> - - <source>import java.io.BufferedInputStream; -import java.io.FileInputStream; -... -public class FileHandler - implements LargeProductQueryHandler { - ... - public byte[] retrieveChunk(String id, long offset, - int length) throws ProductException { - BufferedInputStream in = null; - try { - File f = new File(id); - if (!f.isFile()) throw new ProductException(id - + " isn't a file (anymore?)"); - in = new BufferedInputStream(new FileInputStream(f)); - skip(offset, in); - byte[] buf = read(length, in); - return buf; - } catch (IOException ex) { - throw new ProductException(ex); - } finally { - if (in != null) try { - in.close(); - } catch (IOException ignore) {} - } - } -}</source> - - </subsection> - - <subsection name='Closing Up'> - <p>Because the OODT framework has no idea what data sources a - <code>LargeProductQueryHandler</code> will eventually - consult, what temporary files it may need to clean up, what - network sockets it might need to shut down, and so forth, it - needs some way to indicate to a query handler that's it's - done calling <code>retrieveChunk</code> for a certain - <em>product ID</em>. The <code>close</code> method does this. - </p> - - <p>In our example, <code>close</code> doesn't need to do - anything, but we are obligated to implement it: - </p> - - <source>... -public class FileHandler - implements LargeProductQueryHandler { - ... - public void close(String id) {} -}</source> - </subsection> - - <subsection name='Complete Source Code'> - <p>Here's the complete source file, <code>FileHandler.java</code>:</p> - <source>import java.io.BufferedInputStream; -import java.io.File; -import java.io.FileInputStream; -import java.io.InputStream; -import java.io.IOException; -import java.util.Collections; -import java.util.Iterator; -import java.util.List; -import jpl.eda.product.LargeProductQueryHandler; -import jpl.eda.product.ProductException; -import jpl.eda.xmlquery.LargeResult; -import jpl.eda.xmlquery.QueryElement; -import jpl.eda.xmlquery.XMLQuery; - -public class FileHandler - implements LargeProductQueryHandler { - private static File getFile(XMLQuery q) { - List stack = q.getWhereElementSet(); - if (stack.size() != 3) return null; - QueryElement e = (QueryElement) stack.get(0); - if (!"elemName".equals(e.getRole()) - || !"file".equals(e.getValue())) - return null; - e = (QueryElement) stack.get(2); - if (!"RELOP".equals(e.getRole()) - || !"EQ".equals(e.getValue())) - return null; - e = (QueryElement) stack.get(1); - if (!"LITERAL".equals(e.getRole())) - return null; - File file = new File(e.getValue()); - if (!file.isFile()) return null; - return file; - } - private static String getMimeType(XMLQuery q) { - for (Iterator i = q.getMimeAccept().iterator(); - i.hasNext();) { - String t = (String) i.next(); - if (t.indexOf('*') == -1) return t; - } - return "application/octet-stream"; - } - private static void insert(File file, String type, - XMLQuery q) throws IOException { - String id = file.getCanonicalPath(); - long size = file.length(); - LargeResult lr = new LargeResult(id, type, - /*profileID*/null, /*resourceID*/null, - /*headers*/Collections.EMPTY_LIST, size); - q.getResults().add(lr); - } - public XMLQuery query(XMLQuery q) - throws ProductException { - try { - File file = getFile(q); - if (file == null) return q; - String type = getMimeType(q); - insert(file, type, q); - return q; - } catch (IOException ex) { - throw new ProductException(ex); - } - } - private static void skip(long offset, - InputStream in) throws IOException { - while (offset > 0) - offset -= in.skip(offset); - } - private static byte[] read(int length, - InputStream in) throws IOException { - byte[] buf = new byte[length]; - int numRead; - int index = 0; - int toRead = length; - while (toRead > 0) { - numRead = in.read(buf, index, toRead); - index += numRead; - toRead -= numRead; - } - return buf; - } - public byte[] retrieveChunk(String id, long offset, - int length) throws ProductException { - BufferedInputStream in = null; - try { - File f = new File(id); - if (!f.isFile()) throw new ProductException(id - + " isn't a file (anymore?)"); - in = new BufferedInputStream(new FileInputStream(f)); - skip(offset, in); - byte[] buf = read(length, in); - return buf; - } catch (IOException ex) { - throw new ProductException(ex); - } finally { - if (in != null) try { - in.close(); - } catch (IOException ignore) {} - } - } - public void close(String id) {} -}</source> - </subsection> - </section> - - <section name='Compiling the Code'> - <p>We'll compile this code using the J2SDK command-line tools, - but if you're more comfortable with some kind of Integrated - Development Environment (IDE), adjust as necessary. - </p> - - <p>Let's go back again to the <code>$PS_HOME</code> directory we - made earlier; create the file - <code>$PS_HOME/src/FileHandler.java</code> with the contents - shown above. Then, compile and update the jar file as follows: - </p> - - <source>% <b>javac -extdirs lib \ - -d classes src/FileHandler.java</b> -% <b>ls -l classes</b> -total 8 --rw-r--r-- 1 kelly kelly 2524 25 Feb 15:46 ConstantHandler.class --rw-r--r-- 1 kelly kelly 3163 26 Feb 16:15 FileHandler.class -% <b>jar -uf lib/my-handlers.jar \ - -C classes FileHandler.class</b> -% <b>jar -tf lib/my-handlers.jar</b> -META-INF/ -META-INF/MANIFEST.MF -ConstantHandler.class -FileHandler.class</source> - - <p>We've now got a jar with the <code>ConstantHandler</code> - from the <a href="../qh/">last tutorial</a> and our new - <code>FileHandler</code>. - </p> - </section> - - <section name='Specifying and Running the New Query Handler'> - <p>The <code>$PS_HOME/bin/ps</code> script already has a system - property specifying the <code>ConstantHandler</code>, so we - just need to add the <code>FileHandler</code> to that list. - </p> - - <p>First, stop the product server by hitting CTRL+C (or your - interrupt key) in the window in which it's currently running. - Then, modify the <code>$PS_HOME/bin/ps</code> script to read - as follows: - </p> - - <source>#!/bin/sh -exec java -Djava.ext.dirs=$PS_HOME/lib \ - -Dhandlers=ConstantHandler,FileHandler \ - jpl.eda.ExecServer \ - jpl.eda.product.rmi.ProductServiceImpl \ - urn:eda:rmi:MyProductService</source> - - <p>Then start the server by running - <code>$PS_HOME/bin/ps</code>. If all goes well, the product - server will be ready to answer queries again, this time - passing each incoming <code>XMLQuery</code> to <em>two</em> - different query handlers. - </p> - - <p>Edit the <code>$PS_HOME/bin/pc</code> script once more to - make sure the <code>-out</code> and not the <code>-xml</code> - command-line argument is being used. Let's try querying for a - file: - </p> - - <source>% <b>$PS_HOME/bin/pc "file = /etc/passwd"</b> -nobody:*:-2:-2:Unprivileged User:/:/usr/bin/false -root:*:0:0:System Administrator:/var/root:/bin/sh -daemon:*:1:1:System Services:/var/root:/usr/bin/false -...</source> - - <p>If you like, you can change the <code>-out</code> to - <code>-xml</code> again and examine the XML version. This - time, the product data isn't in the XMLQuery object. - </p> - </section> - - <section name="What's the Difference?"> - <p>On the client side, the interface to get product results in - <code>LargeResult</code>s versus regular <code>Result</code>s - is identical. The client calls <code>getInputStream</code> to - get a binary stream to read the product data. - </p> - - <p>There is a speed penalty for large results. What - <code>Result.getInputStream</code> returns is an input stream - to product data already contained in the XMLQuery. It's a - stream to a buffer already in the client's address space, so - it's nice and fast. - </p> - - <p><code>LargeResult</code> overrides the - <code>getInputStream</code> method to instead return an input - stream that repeatedly makes calls back to the product - server's <code>retrieveChunk</code> method. Since the product - is <em>not</em> already in the local address space of the - client, getting large products is a bit slower. To - compensate, the input stream actually starts a background - thread to start retrieving chunks of the product ahead of the - product client, up to a certain point (we don't want to run - out of memory again). - </p> - - <p>On the server side, the difference is in programming - complexity. Creating a <code>LargeProductQueryHandler</code> - requires implementing three methods instead of just one. You - may have to clean up temporary files, close network ports, or - do other cleanup. You may even have to guard against clients - that present specially-crafted product IDs that try to - circumvent access controls to products. - </p> - - <p><code>LargeResult</code>s are more general, and will work for - any size product, from zero bytes on up. And you can even mix - and match: a <code>LargeProductQueryHandler</code> can add - regular <code>Result</code>s to an XMLQuery as well as - <code>LargeResult</code>s. You might program some logic that, - under a certain threshold, to return regular - <code>Result</code>s for small sized products, and - <code>LargeResult</code>s for anything bigger than small. - </p> - </section> - - <section name='Conclusion'> - <p>In this tutorial, we implemented a - <code>LargeProductQueryHandler</code> that served large - products. In this case, large could mean zero bytes (empty - products) up to gargantuan numbers of bytes. This handler - queried for files in the product server's filesystem, which is - a bit insecure so you might want to terminate the product - server as soon as possible. We also learned that what the - advantages and disadvantages were between regular product - results and large product results, and that - <code>LargeProductQueryHandler</code>s can use - <code>LargeResult</code> objects in addition to regular - <code>Result</code> objects. - </p> - - <p>If you've also completed the <a href="../ps">Your First - Product Service</a> tutorial and the <a - href="../qh/">Developing a Query Handler</a> tutorial, you - are now a master of the OODT Product Service. - Congratulations! - </p> - </section> - </body> -</document>
http://git-wip-us.apache.org/repos/asf/oodt/blob/098cc4fa/product/src/site/xdoc/tutorials/ps/index.xml ---------------------------------------------------------------------- diff --git a/product/src/site/xdoc/tutorials/ps/index.xml b/product/src/site/xdoc/tutorials/ps/index.xml deleted file mode 100755 index 91bfc00..0000000 --- a/product/src/site/xdoc/tutorials/ps/index.xml +++ /dev/null @@ -1,481 +0,0 @@ -<?xml version="1.0" encoding="UTF-8"?> -<!-- - Licensed to the Apache Software Foundation (ASF) under one or more - contributor license agreements. See the NOTICE file distributed with - this work for additional information regarding copyright ownership. - The ASF licenses this file to You under the Apache License, Version 2.0 - (the "License"); you may not use this file except in compliance with - the License. You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. ---> -<document> - <properties> - <title>Your First Product Service</title> - <author email="[email protected]">Sean Kelly</author> - </properties> - - <body> - <section name="Your First Product Service"> - <p>This tutorial introduces starting a basic product server. - This product server will be capable of accepting queries, but - will not actually respond with any data. By completing this - tutorial, you'll have a working product server in which you - can install more complex logic to actually handle product - requests. - </p> - </section> - - <section name="The Product Service"> - <p>The OODT Product Service is a remotely accessible software - component that enables you to retrieve products, which can be - any kind of data. In OODT, a <em>product client</em> passes a - <em>query</em> into a known product server. The product - server delegates that query to its installed <em>query - handlers</em>; each one gets a chance at satisfying the query - with requested data. Query handlers are the interfaces between - the generic OODT framework and your system-specific data - stores. They have the job of understanding the passed in - query, finding or even synthesizing the matching product, - applying conversions to Internet-standard formats, and - returning results. The product service then collects all the - matching products from the query handlers and returns them to - the product client. - </p> - - <p>To deploy a product server, you need to come up with query - handlers that interface to your site- or discipline-specific - data stores. Then, start a product server and inform it of - what query handlers to use. - </p> - - <subsection name="Delegation Model"> - <p>The OODT product service <em>delegates</em> incoming - queries to zero or more query handlers. In the case of zero - query handlers, the product service always replies to every - query with "zero matches." Otherwise, the query handlers - get a chance to satisfy the query, and may or may not add - matching products to the result. - </p> - - <p>The following class diagram demonstrates this delegation model:</p> - - <img src="../../images/delegation.png" alt="Delegation Class Diagram" /> - - <p>Here, a product client calls a server to process a query - for products. The server delegates to query handlers, which - are Java objects that implement the - <code>QueryHandler</code> interface. Two query handlers in - this diagram, <code>MyHandler</code> and - <code>MyOtherHandler</code> can both try to satisfy the - query by adding matching products to the total result. They - can each add more than one matching product, just one, or - none at all. The server then returns the matches, if any, - to the client. - </p> - </subsection> - - <subsection name="Large Products"> - <p>In OODT, a query contains its matching products. When a - client passes a query to a product server, the query object - returns to the client <em>with matching products embedded in - it</em>. This can, however, make query objects too large to - be comfortably passed around a network of OODT services (query - objects must reside completely in memory). In this case, a - special extension of a <code>QueryHandler</code>, a - <code>LargeProductQueryHandler</code>, can instead place a - <em>reference</em> to the product data in the query. - </p> - <p>To product clients, the difference is invisible: the - product data is still accessed from the query object the - same way. As a developer of product services, though, you - may need to decide which kind of query handler to make: - regular or large. - </p> - </subsection> - - <subsection name="Communicating with a Product Service"> - <p>The product service is a remotely accessible object. - Therefore, product clients access it with a remote object - access protocol. Currently, OODT supports RMI and CORBA. You - can also access product services with HTTP; in this case, a - proxy object provides the HTTP interface while internally it - accesses a product service with RMI or CORBA. - </p> - - <p>For this tutorial, we'll use RMI because it's enormously - less complex than CORBA. - </p> - </subsection> - </section> - - <section name="Making the Staging Area"> - <p>To start a product service, we'll create a directory - structure that will hold software components (jar files) as - well as scripts that will simplify the usually over-long Java - command lines. (Note that these examples are for Mac OS X, - Linux, or other Unix-like systems. Windows users will have to - adapt.) - </p> - - <p>Let's start by making a directory hierarchy for our product - service called <code>ps</code> (this example uses a C-style - shell <code>csh</code>, if you're using <code>bash</code> or - another style shell, substitute the appropriate commands). - </p> - - <source>% <b>mkdir ps</b> -% <b>cd ps</b> -% <b>setenv PS_HOME `pwd`</b> -% <b>mkdir bin lib</b> -% <b>ls -RF $PS_HOME</b> -bin/ lib/ - -/Users/kelly/tmp/ps/bin: - -/Users/kelly/tmp/ps/lib:</source> - - <p>Note that we're using an environment variable - <code>PS_HOME</code> to contain the path of the directory - we're using to hold everything. We'll use this environment - variable as we develop the scripts to launch the product service. - </p> - </section> - - <section name="The RMI Registry"> - <p>Since we're using Remote Method Invocation (RMI) for this - tutorial, we'll need to start an RMI Registry. An RMI - Registry serves as a catalog that maps between named objects, - such as your product server, to the appropriate network - address and port where the object can be located. Your - product client will use the RMI registry to locate the product - server so it can connect to the product server and communicate with it. - </p> - - <subsection name="Collecting the RMI Registry Components"> - <p>To start an RMI Registry, you'll need the following components:</p> - - <ul> - <li><a href="/edm-commons/">EDM Common Components</a>. - These are common utilities used by every OODT - service.</li> <li><a href="/grid-product/">Grid Product - Service</a>. This is the product service, product client, - query handler interface, and related classes.</li> <li><a - href="/rmi-registry/">OODT RMI Registry</a>. This is the - actual RMI registry.</li> - </ul> - - <p>Download each component's binary distribution, unpack each - one, and take collect the jar files into the - <code>lib</code> directory. For example: - </p> - - <source>% <b>cp /tmp/edm-commons-2.2.5/*.jar $PS_HOME/lib</b> -% <b>cp /tmp/grid-product-3.0.3/*.jar $PS_HOME/lib</b> -% <b>cp /tmp/rmi-registry-1.0.0/*.jar $PS_HOME/lib</b> -% <b>ls -l $PS_HOME/lib</b> -total 312 --rw-r--r-- 1 kelly kelly 149503 24 Feb 14:06 edm-commons-2.2.5.jar --rw-r--r-- 1 kelly kelly 120844 24 Feb 14:07 grid-product-3.0.3.jar --rw-r--r-- 1 kelly kelly 8055 24 Feb 14:07 rmi-registry-1.0.0.jar</source> - </subsection> - - <subsection name="Writing the RMI Script"> - <p>To keep from having to type long Java command lines, we'll - create a simple shell script that will start the RMI - registry. We'll call it <code>rmi-reg</code> and stick it - in the <code>bin</code> directory. - </p> - - <p>Here's the <code>rmi-reg</code> script:</p> - <source>#!/bin/sh -exec java -Djava.ext.dirs=$PS_HOME/lib \ - gov.nasa.jpl.oodt.rmi.RMIRegistry</source> - - <p>This script tells the Java virtual machine to find - extension jars in the directory <code>$PS_HOME/lib</code>. It - then says that the main class to execute is - <code>gov.nasa.jpl.oodt.rmi.RMIRegistry</code>. - </p> - - <p>Go ahead and make this script executable and start the RMI - Registry. In another window (with the appropriate setting of - <code>PS_HOME</code>), run - <code>$PS_HOME/bin/rmi-reg</code>. You should see output - similar to the following: - </p> - - <source>% <b>chmod 755 $PS_HOME/bin/rmi-reg</b> -% <b>$PS_HOME/bin/rmi-reg</b> -Thu Feb 24 14:10:25 CST 2005: no objects registered</source> - - <p>The RMI Registry is now running. Every two minutes it will - display an update of all registered objects. Naturally, we - don't have any product service running right now, so it will - say <code>no objects registered</code>. Go ahead and ignore - this window for now. It's time to start our product server. - </p> - </subsection> - </section> - - <section name="The Product Server"> - <p>With an RMI Registry in place, we're ready to start our - product server. As with the RMI Registry, we'll need the - software components and to make a script to launch it. - </p> - - <subsection name="Collecting the Product Server Components"> - <p>We already have two of the components needed to start the - product server, <code>edm-commons</code> and - <code>grid-product</code>. We need two more: - </p> - - <ul> - <li><a href="/edm-query/">EDM Query Expression</a>. This - component encapsulates the implementation of an OODT query - and also contains some product retrieval utilities.</li> - <li><a href="http://ws.apache.org/xmlrpc";>Apache - XML-RPC</a>. This is used internally by OODT services. - Download version 1.1, not a later version! If you prefer, - you can <a - href="http://ibiblio.org/maven/xmlrpc/jars/xmlrpc-1.1.jar";>fetch - the jar file directly</a>.</li> - </ul> - - <p>As before, put these jars into the - <code>$PS_HOME/lib</code> directory: - </p> - - <source>% <b>ls -l $PS_HOME/lib</b> -total 376 --rw-r--r-- 1 kelly kelly 149503 24 Feb 14:06 edm-commons-2.2.5.jar --rw-r--r-- 1 kelly kelly 43879 24 Feb 14:35 edm-query-2.0.2.jar --rw-r--r-- 1 kelly kelly 120844 24 Feb 14:07 grid-product-3.0.3.jar --rw-r--r-- 1 kelly kelly 8055 24 Feb 14:07 rmi-registry-1.0.0.jar --rw-r--r-- 1 kelly kelly 53978 24 Feb 14:35 xmlrpc-1.1.jar</source> - </subsection> - - <subsection name="Writing the Product Server Script"> - <p>To launch the product server, we'll create a script called - <code>ps</code> in the <code>$PS_HOME/bin</code> directory. - Here's its contents: - </p> - - <source>#!/bin/sh -exec java -Djava.ext.dirs=$PS_HOME/lib \ - jpl.eda.ExecServer \ - jpl.eda.product.rmi.ProductServiceImpl \ - urn:eda:rmi:MyProductService</source> - - <p>Like with the RMI server, this tells Java where to find - extension jars (<code>$PS_HOME/lib</code>). The main class - is <code>jpl.eda.ExecServer</code>, this is a framework - class from <code>edm-commons</code> that provides basic - start-up functions for a variety of services. In this case, - the service is - <code>jpl.eda.product.rmi.ProductServiceImpl</code>; this is - the name of the class that provides the RMI version of the - OODT product service. We then pass in one final - command-line argument, - <code>urn:eda:rmi:MyProductService</code>. This names the product service. - </p> - </subsection> - - <subsection name="What's in a Name?"> - <p>The product service registers itself using a name provided - on the command-line, in this case, - <code>urn:eda:rmi:MyProductService</code>. Let's take apart - the name and see how it works. - </p> - - <p>If you're familiar with web standards, you can see that the - name is a Uniform Resource Name (URN), since it starts with - <code>urn:</code>. The OODT Framework uses URNs to identify - services and other objects. The <code>eda:</code> tells - that the name is part of the Enterprise Data Architecture - (EDA) namespace. (EDA was the name of a project related to - OODT that was merged with OODT. For now, just always use - <code>eda:</code> in your URNs.) - </p> - - <p>Next comes <code>rmi:</code>. This is a special flag for - the OODT services that tells that we're using a name of an - RMI-accessible object. The OODT framework will know to use - an RMI Registry to register the server. - </p> - - <p>Finally is <code>MyProductService</code>. This is the - actual name used in the RMI Registry. You can call your - product service anything you want. For example, suppose you - have three product servers; one in the US, one in Canada, - and one in Australia. You might name them: - </p> - - <ul> - <li><code>urn:eda:rmi:US</code></li> - <li><code>urn:eda:rmi:Canada</code></li> - <li><code>urn:eda:rmi:Australia</code></li> - </ul> - - <p>Or you might prefer to use ISO country codes. Or you might - name them according to the kinds of products they serve, - such as <code>urn:eda:rmi:Biomarkers</code> or - <code>urn:eda:rmi:BusniessForecasts</code>. - </p> - - <p>The RMI Registry will happily re-assign a name if one's - already in use, so when deploying your own product servers, - be sure to give each one a unique name. - </p> - </subsection> - - <subsection name="Launching the Product Server"> - <p>Make the <code>ps</code> script executable and start the - product server at this time. Do this in a separate window - with the appropriate setting of <code>PS_HOME</code>: - </p> - - <source>% <b>chmod 755 $PS_HOME/bin/ps</b> -% <b>$PS_HOME/bin/ps</b> -Object context ready; delegating to: [jpl.eda.object.jndi.RMIContext@94257f]</source> - - <p>The product service is now running and ready to accept - product queries. Since we didn't tell it what query - handlers to use, it will always respond with zero matching - products. That may not be interesting, but it's a good test - to see if we can at least launch a product server. Now, - let's launch a product client and query it. - </p> - </subsection> - </section> - - <section name="Querying the Product Server"> - <p>To query the product server, we use a product client. The - Java class <code>jpl.eda.product.ProductClient</code> provides - the API for your own programs to query for and retrieve - products. But it's also an executable class, so we can run it - from the command-line in order to test our product server. - However, let's again make a script to make invoking it a bit - easier. - </p> - - <p>We'll call the script <code>pc</code> for "product client," - and it will take a single command line argument, which will be - the <em>query expression</em> to pass into the product server. - Query expressions define the constraints on the kinds of - products we want to achieve. Since the product server we've - set up will always respond with zero products, though, we can - pass in any syntactically valid query expression. - </p> - - <p>Here's the script:</p> - - <source>#!/bin/sh -if [ $# -ne 1 ]; then - echo "Usage: `basename $0` <query-expression>" 1>&2 - exit 1 -fi - -exec java -Djava.ext.dirs=$PS_HOME/lib \ - jpl.eda.product.ProductClient \ - -out \ - urn:eda:rmi:MyProductService \ - "$1"</source> - - - <p>This script checks to make sure there's exactly one - command-line argument, the query expression. If there isn't, - it prints a helpful usage message to the standard error - stream, as is Unix tradition. Otherwise, it will execute the - <code>jpl.eda.product.ProductClient</code> class with Java. - When executed, this class expects three command-line arguments: - </p> - - <ol> - <li><code>-out</code> or <code>-xml</code>. OODT uses XML to - represent the query that it passes to and receives back from - a product server. With <code>-out</code>, the product - client will write to the standard output the raw product - data. With <code>-xml</code>, you'll instead see the XML - representation of the query (with any embedded matching - products) instead. - </li> - - <li>The name of the product service to contact. In this case, - we're using the one we started earlier, registered under the - name <code>urn:eda:rmi:MyProductService</code>. - </li> - - <li>The query expression.</li> - </ol> - - <p>Now we can make this script executable and run it:</p> - - <source>% <b>chmod 755 $PS_HOME/bin/pc</b> -<b>$PS_HOME/bin/pc "x = 3"</b> -Object context ready; delegating to: [jpl.eda.object.jndi.RMIContext@c79809] -No matching results</source> - - <p>Although not terribly exciting, this is good news. Here's - what happened: - </p> - - <ol> - <li>The product client created a query object (of class - <code>jpl.eda.xmlquery.XMLQuery</code> from the - <code>edm-query</code> component) out of the string query - <code>x = 3</code>. - </li> - - <li>It asked the RMI Registry to tell it where (network - address) it could find the product service named - <code>MyProductService</code>. - </li> - - <li>After getting the response back from the RMI Registry, it - then contacted the product service over a network connection - (even if to the same local system) and asked it to handle - the query, passing the query object. - </li> - - <li>The product service, having no query handlers to which to - delegate, merely returned the query object unmodified over - the network connection. - </li> - - <li>The product client, having no product to write to the - standard output (as indicated by the <code>-out</code> - argument), wrote the diagnostic message <code>No matching - results</code>. - </li> - </ol> - - <p>You can make this example slightly more interesting by - changing the <code>-out</code> in the <code>pc</code> script - to <code>-xml</code>. Now, when you run it, you'll see an XML - document describing the query. One of the pertinent sections - to note is: - </p> - - <source>...<queryResultSet/>...</source> - - <p>This empty XML element means that there were no results.</p> - </section> - - <section name="Conclusion"> - <p>By following this tutorial, you've started both an RMI - Registry and a basic product server. You've queried that - product server to insure that you can communicate with it. In - later tutorials, you'll build on this product server by adding - a query handler to it and returning actual product data. - </p> - </section> - - </body> -</document> http://git-wip-us.apache.org/repos/asf/oodt/blob/098cc4fa/product/src/site/xdoc/tutorials/qh/index.xml ---------------------------------------------------------------------- diff --git a/product/src/site/xdoc/tutorials/qh/index.xml b/product/src/site/xdoc/tutorials/qh/index.xml deleted file mode 100755 index b8e34f6..0000000 --- a/product/src/site/xdoc/tutorials/qh/index.xml +++ /dev/null @@ -1,704 +0,0 @@ -<?xml version="1.0" encoding="UTF-8"?> -<!-- - Licensed to the Apache Software Foundation (ASF) under one or more - contributor license agreements. See the NOTICE file distributed with - this work for additional information regarding copyright ownership. - The ASF licenses this file to You under the Apache License, Version 2.0 - (the "License"); you may not use this file except in compliance with - the License. You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - - Unless required by applicable law or agreed to in writing, software - distributed under the License is distributed on an "AS IS" BASIS, - WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - See the License for the specific language governing permissions and - limitations under the License. ---> -<document> - <properties> - <title>Developing a Query Handler</title> - <author email="[email protected]">Sean Kelly</author> - </properties> - - <body> - <section name="Developing a Query Handler"> - <p>In the <a href="../ps/">last tutorial</a>, we started a - product server. But this wasn't a very useful product server; - it could answer queries but always respond with no results. - That's because it had no query handlers. Query handlers have - the responsibility of actually handling product queries. In - this tutorial, we'll develop a query handler, install it into - our product server, and query it to see if it works. - </p> - - <p>To do this tutorial, you'll need mastery of two things:</p> - <ul> - <li>Using the <code>XMLQuery</code> class. Follow the <a - href="/edm-query/tutorial/">query expression tutorail</a> - now if you're not familiar with it. - </li> - - <li>Running and querying a product server. Follow the <a - href="../ps/">Your First Product Server</a> tutorial to - get your product server up and running. In this tutorial, - we'll build on that product server, so it's especially - important to have it in good shape. - </li> - </ul> - </section> - - <section name="Serving Up Constants"> - <p>Product servers delegate to query handlers. It's the job of - query handlers to interpret incoming queries (expressed as - <code>XMLQuery</code> objects), search for, retrieve, convert, - or synthesize matching product results, adorn the - <code>XMLQuery</code> object with <code>Result</code> objects, - and return the modified query. At that point the OODT - framework takes over again and tries other installed query - handlers, eventually returning the completed - <code>XMLQuery</code> back to the product client that made the - query in the first place. - </p> - - <p>We'll make a query handler that serves mathematical - constants. Have you ever been in a position where you needed, - say, the value of the third Flajolet number or perhaps - Zeta(9)? No? Well, just pretend for now you did. What we'll - do is develop a query handler for a product server that will - serve values of various mathematical constants. - </p> - - <p>The approach we'll take has three simple steps:</p> - - <ol> - <li>Get some handy constants.</li> - <li>Define the query expression.</li> - <li>Write a query handler. The query handler will: - <ol> - <li>Examine the query expression to see if it's a request - for a constant, and if so, what constant is - requested. - </li> - <li>Examine the query's list of acceptable MIME types.</li> - <li>If both check out, look up the desired constant's value.</li> - <li>If found, add it as a <code>Result</code> in the <code>XMLQuery</code>.</li> - </ol> - </li> - </ol> - </section> - - <section name="Writing the Code"> - <p>In this section, we'll build up the query handler source code - in pieces, examining each piece thoroughly. We'll then - present the entire source file. - </p> - - <subsection name='Gathering Handy Constants'> - <p>The wonderful world of science and mathematics is replete - with useful constant values. For this example, let's just pick three: - </p> - - <ul> - <li><var>pi</var> = 3.14159265...</li> - <li><var>e</var> = 2.7182818285...</li> - <li><var>gamma</var> = 0.577215664...</li> - </ul> - - <p>In Java code, we can set up those values as a - <code>Map</code> in a static field. Thus we start forming our - source file, <code>ConstantHandler.java</code>: - </p> - - <source>import java.util.concurrent.ConcurrentHashMap; -import java.util.Map; -import jpl.eda.product.QueryHandler; -public class ConstantHandler - implements QueryHandler { - private static final Map CONSTANTS = new ConcurrentHashMap(); - static { - CONSTANTS.put("pi", "3.14159265..."); - CONSTANTS.put("e", "2.7182818285..."); - CONSTANTS.put("gamma", "0.577215664..."); - } -}</source> - - <p>As you can see, we're storing both the constant name and its - value as <code>java.lang.String</code> objects. - </p> - </subsection> - - <subsection name='Defining the Query Expression'> - <p>Recall that the <code>XMLQuery</code> class can use parsed - queries (where it generates postfix boolean stacks) or - unparsed ones. While unparsed ones are easier, we'll go with - parsed ones to demonstrate how on the server-side you deal - with those postfix stacks. - </p> - - <p>Using the XMLQuery's expression language, we'll look for - queries of the form:</p> - - <p><code>constant = <var>name</var></code></p> - - <p>where <var>name</var> is the name of a constant. That will - form a postfix "where" stack with exactly three - <code>QueryElement</code> objects on it: - </p> - - <ol> - <li>The first (top) <code>QueryElement</code> will have role = - <code>elemName</code> and value = <code>constant</code>. - </li> - - <li>The second (middle) <code>QueryElement</code> will have - role = <code>LITERAL</code> and a value equal to the - constant <var>name</var>. - </li> - - <li>The third (bottom) <code>QueryElement</code> will have - role = <code>RELOP</code> and value = <code>EQ</code>. - </li> - </ol> - - <p>If we get any other kind of stack, we'll reject it and return - no matching results. That's reasonable behavior; after all, a - query for <code>donutsEaten > 5 AND RETURN = - episodeNumber</code> may be handled by a - <code>SimpsonsEpisodeQueryHandler</code> that's <em>also</em> - installed in the same product server. - </p> - - <p>We'll define a utility method, <code>getConstantName</code>, - that will take the <code>XMLQuery</code>, check for the - postfix "where" stack as described, and return the matching - constant <var>name</var>. If it gets a stack whose structure - doesn't match, it will return <code>null</code>. We'll add - this method to our <code>ConstantHandler.java</code> file: - </p> - - <source>import java.util.List; -import jpl.eda.xmlquery.XMLQuery; -import jpl.eda.xmlquery.QueryElement; -... -public class ConstantHandler - implements QueryHandler { - ... - private static String getConstantName(XMLQuery q) { - List stack = q.getWhereElementSet(); - if (stack.size() != 3) return null; - QueryElement e = (QueryElement) stack.get(0); - if (!"elemName".equals(e.getRole()) - || !"constant".equals(e.getValue())) - return null; - e = (QueryElement) stack.get(2); - if (!"RELOP".equals(e.getRole()) - || !"EQ".equals(e.getValue())) - return null; - e = (QueryElement) stack.get(1); - if (!"LITERAL".equals(e.getRole())) - return null; - return e.getValue(); - } -}</source> - - <p>Here, we first check to make sure there's exactly three - elements, returning null if not. There's no need to go further. - </p> - - <p>Assuming there's three elements, the code then checks the - topmost element. For an expression <code>constant = - <var>name</var></code>, the topmost element will have role - <code>elemName</code> and value <code>constant</code>. If - neither condition is true, we return null right away. No need - to check further. - </p> - - <p>If the topmost element checks out, we then check the - bottommost element. For <code>constant = - <var>name</var></code>, the bottom element is generated from - the equals sign. It will have role <code>RELOP</code> - (relational operator) and value <code>EQ</code>, meaning - "equals". - </p> - - <p>If it checks out, all we have to do is check the middle - element. The infix expression <code>constant = - <var>name</var></code> generates a postfix middle element of - <var>name</var> as the value, with a role of - <code>LITERAL</code>. We make sure it's <code>LITERAL</code>. - If not, we're done; it's not a valid expression for our query - handler. - </p> - - <p>But if so, then the value of that query element is the name - of the desired constant. So we return it, regardless of what - it is. - </p> - </subsection> - - <subsection name='Checking for Acceptable MIME Types'> - <p>Since all of our mathematical constants are strings, we'll - say that the result MIME type of our products is - <code>text/plain</code>. That means that any incoming - <code>XMLQuery</code> must include any of the following MIME types: - </p> - <ol> - <li><code>text/plain</code></li> - <li><code>text/*</code></li> - <li><code>*/*</code></li> - </ol> - <p>All of these match <code>text/plain</code>, which is the only - product type we're capable of serving. (In your own product - servers, you might have more complex logic; for example, you - could write code to draw the numbers into an image file if the - requested type is <code>image/jpeg</code> ... but I wouldn't - want to.) - </p> - <p>To support this in our query handler, we'll write another - utility method. It'll be called - <code>isAcceptableType</code>, and it will take the - <code>XMLQuery</code> and examine it to see what MIME types - are acceptable to the caller. If it finds any of the ones in - the above list, it will return <code>true</code>, and the - caller can continue to process the query. If not, it will - return <code>false</code>, and the query handler will stop - processing and return the <code>XMLQuery</code> unadorned with - any results. - </p> - <p>Here's the code:</p> - - <source>import java.util.Iterator; -... -public class ConstantHandler - implements QueryHandler { - ... - private static boolean isAcceptableType(XMLQuery q) { - List mimes = q.getMimeAccept(); - if (mimes.isEmpty()) return true; - for (Iterator i = mimes.iterator(); i.hasNext();) { - String type = (String) i.next(); - if ("text/plain".equals(type) - || "text/*".equals(type) - || "*/*".equals(type)) return true; - } - return false; - } -}</source> - - <p>Here, we check if the list of acceptable MIME types is empty. - An empty list is the same as saying <code>*/*</code>, so that - automatically says we've got an acceptable type. For a - non-empty list, we go through each type one-by-one. If it's - any of the strings <code>text/plain</code>, - <code>text/*</code>, or <code>*/*</code>, then that's an - acceptable type. - </p> - - <p>However, if we get through the entire list and we don't find - any type that the user wants that we can provide, we return - <code>false</code>. The query handler will check for a - <code>false</code> value and return early from handling the - query, leaving the <code>XMLQuery</code> untouched. - </p> - </subsection> - - <subsection name='Inserting the Result'> - <p>Assuming the query handler has found an acceptable MIME type, - and has found a valid query and the name of the desired - constant, it can lookup the constant in the - <code>CONSTANTS</code> map. And assuming it finds a matching - constant in that map, it can insert the value as a - <code>Result</code> object. - </p> - - <p>To insert the constant's value, we'll develop yet another - utility method, this time called <code>insert</code>. This - method will take the name of the constant, its value, and the - <code>XMLQuery</code>. It will add a <code>Result</code> - object to the <code>XMLQuery</code>. When the query handler - returns this modified <code>XMLQuery</code> object, the - framework will return it to the product client, which can then - display the matching result. - </p> - - <p><code>Result</code> objects can also have optional - <code>Header</code> objects that serve as "column headings" - for tabular like results. Our result isn't tabular, it's just - a single value, but we'll add a heading anyway just to - demonstrate how it's done. (You could argue that it's a - one-by-one table, too!) The header's name will be the same as - the constant's name; the data type will be <code>real</code> - and the units will be <code>none</code>. - </p> - - <p>Here's the code:</p> - - <source>import java.util.Collections; -import jpl.eda.xmlquery.Header; -import jpl.eda.xmlquery.Result; -... -public class ConstantHandler - implements QueryHandler { - ... - private static void insert(String name, - String value, XMLQuery q) { - Header h = new Header(name, "real", "none"); - Result r = new Result(name, "text/plain", - /*profileID*/null, /*resourceID*/null, - Collections.singletonList(h), - value, /*classified*/false, Result.INFINITE); - q.getResults().add(r); - } -}</source> - - <p>In this method, we first create the header. Then we create - the result; the result's ID (which differentiates it from - other results in the same <code>XMLQuery</code> is just the - name of the constant. Its MIME type is - <code>text/plain</code>. We set the profile ID and resource - ID fields to <code>null</code>, as recommended back in the <a - href="/edm-query/tutorial/">XMLQuery Tutorial</a>. Then we - add our sole header. Then we add the mathematical constant's - value. Finally, this constant isn't classified, so we set the - classified flag to <code>false</code>. Also, these - mathematical constants should be valid forever, so we set the - validity period to <code>Result.INFINITE</code>, a special - value that means a never-ending validity period. - </p> - </subsection> - - <subsection name='Handling the Query'> - <p>With all of these utility methods in hand, it's easy to - handle the query now. The - <code>jpl.eda.product.QueryHandler</code> interface - specifies a single method that we must implement, - <code>query</code>. This method accepts an - <code>XMLQuery</code> object and returns an - <code>XMLQuery</code> object. The returned one may or may - not be adorned with matching results. - </p> - - <p>Here's what we have to do:</p> - <ol> - <li>Get the constant name with <code>getConstantName</code>. - If we get <code>null</code>, it means the query's not of - the form <code>constant = <var>name</var></code>, so we - ignore it. - </li> - - <li>See if the user's willing to accept a - <code>text/plain</code> MIME type. If not, we ignore this query. - </li> - - <li>Find the constant in our <code>CONSTANTS</code> map. If - it's not there, we ignore this query. - </li> - - <li>Insert the constant's value into the <code>XMLQuery</code>.</li> - - <li>Returned the modified <code>XMLQuery</code>.</li> - </ol> - - <p>The source:</p> - - <source>public class ConstantHandler - implements QueryHandler { - ... - public XMLQuery query(XMLQuery q) { - String name = getConstantName(q); - if (name == null) return q; - if (!isAcceptableType(q)) return q; - String value = (String) CONSTANTS.get(name); - if (value == null) return q; - insert(name, value, q); - return q; - } -}</source> - - </subsection> - - <subsection name='Complete Source Code'> - <p>Here is the complete source file, - <code>ConstantHandler.java</code>:</p> - - <source>import java.util.Collections; -import java.util.concurrent.ConcurrentHashMap; -import java.util.Iterator; -import java.util.List; -import java.util.Map; -import jpl.eda.product.QueryHandler; -import jpl.eda.xmlquery.Header; -import jpl.eda.xmlquery.Result; -import jpl.eda.xmlquery.QueryElement; -import jpl.eda.xmlquery.XMLQuery; - -public class ConstantHandler - implements QueryHandler { - private static final Map CONSTANTS = new ConcurrentHashMap(); - static { - CONSTANTS.put("pi", "3.14159265..."); - CONSTANTS.put("e", "2.7182818285..."); - CONSTANTS.put("gamma", "0.577215664..."); - } - private static String getConstantName(XMLQuery q) { - List stack = q.getWhereElementSet(); - if (stack.size() != 3) return null; - QueryElement e = (QueryElement) stack.get(0); - if (!"elemName".equals(e.getRole()) - || !"constant".equals(e.getValue())) - return null; - e = (QueryElement) stack.get(2); - if (!"RELOP".equals(e.getRole()) - || !"EQ".equals(e.getValue())) - return null; - e = (QueryElement) stack.get(1); - if (!"LITERAL".equals(e.getRole())) - return null; - return e.getValue(); - } - private static boolean isAcceptableType(XMLQuery q) { - List mimes = q.getMimeAccept(); - if (mimes.isEmpty()) return true; - for (Iterator i = mimes.iterator(); i.hasNext();) { - String type = (String) i.next(); - if ("text/plain".equals(type) - || "text/*".equals(type) - || "*/*".equals(type)) return true; - } - return false; - } - private static void insert(String name, - String value, XMLQuery q) { - Header h = new Header(name, "real", "none"); - Result r = new Result(name, "text/plain", - /*profileID*/null, /*resourceID*/null, - Collections.singletonList(h), - value, /*classified*/false, Result.INFINITE); - q.getResults().add(r); - } - public XMLQuery query(XMLQuery q) { - String name = getConstantName(q); - if (name == null) return q; - if (!isAcceptableType(q)) return q; - String value = (String) CONSTANTS.get(name); - if (value == null) return q; - insert(name, value, q); - return q; - } -}</source> - - <p>How should you go about compiling this and installing it in - a product server? Read on! - </p> - </subsection> - </section> - - <section name="Compiling the Code"> - <p>We'll compile this code using the J2SDK command-line tools, - but if you're more comfortable with some kind of Integrated - Development Environment (IDE), adjust as necessary. - </p> - - <p>First, let's go back to the <code>$PS_HOME</code> directory - we made earlier and make directories to hold both the source - code and classes that we'll compile from it: - </p> - - <source>% <b>cd $PS_HOME</b> -% <b>mkdir classes src</b></source> - - <p>Then, create <code>$PS_HOME/src/ConstantHandler.java</code> using - your favorite text editor (or by cutting and pasting the source - from this page, or whatever). Finally, compile the file as follows: - </p> - - <source>% <b>javac -extdirs lib \ - -d classes src/ConstantHandler.java</b> -% ls -l classes -total 4 --rw-r--r-- 1 kelly kelly 2524 25 Feb 15:46 ConstantHandler.class</source> - - <p>The <code>javac</code> command is the Java compiler. The - <code>-extdirs lib</code> arguments tell the compiler where to - find extension jars. In this case, the code references things - defined in edm-query-2.0.2.jar and grid-product-3.0.3.jar. - The <code>-d classes</code> tells where compiled classes - should go. - </p> - - <p>Next, make a jar file that contains your compiled class:</p> - - <source>% <b>jar -cf lib/my-handlers.jar \ - -C classes ConstantHandler.class</b> -% <b>jar -tf lib/my-handlers.jar</b> -META-INF/ -META-INF/MANIFEST.MF -ConstantHandler.class</source> - - <p>We now have a new jar file of our own creation in the - <code>$PS_HOME/lib</code> directory; this means that the - product server will be able to find out new query handler. - All we have to do now is tell our product server about it. - </p> - </section> - - <section name='Specfying the Query Handler'> - <p>Query handlers aren't really <em>installed</em> into product - servers. What you do is tell the product server what query - handlers you want it to use by naming their classes. The - product server will instantiate an object of each class and, - as queries come in, it will delegate queries to each - instantiated query handler. - </p> - - <p>To tell a product server what query handlers to instantiate, - you specify a system property called <code>handlers</code>. - You set this property to a comma-separated list of class - names. These should be fully-qualified class names (with - package prefixes, if you used packages when making your query - handlers), separated by commas. In this tutorial, we made - just one query handler, and we didn't put it into a package, - so we'll just use <code>ConstantHandler</code>. - </p> - - <p>First, stop any product server you have running now by - pressing CTRL+C (or whatever your interrupt key is) in the - window that was running the product server. Next, modify the - <code>$PS_HOME/bin/ps</code> file so it reads as follows: - </p> - - <source>#!/bin/sh -exec java -Djava.ext.dirs=$PS_HOME/lib \ - -Dhandlers=ConstantHandler \ - jpl.eda.ExecServer \ - jpl.eda.product.rmi.ProductServiceImpl \ - urn:eda:rmi:MyProductService</source> - - <p>We specified a system property on the command line using - Java's <code>-D</code> option. This defines the system - property <code>handlers</code> as having the value - <code>ConstantHandler</code>. Finally, start the product - server again by running <code>$PS_HOME/bin/ps</code>. - </p> - </section> - - <section name='Querying for Constants'> - <p>Once again, edit the <code>$PS_HOME/bin/pc</code> script and - change <code>-xml</code> back to <code>-out</code> so that - instead of the XML output we'll see the raw product data. - Then run it and see what happens: - </p> - - <source>% <b>$PS_HOME/bin/pc 'constant = pi'</b> -3.14159265...% </source> - - <p>Because the raw product data was the string - <code>3.14159265...</code> without any trailing newline, the - shell's prompt appeared right at the end of the product - result. You might try piping the output of the above command - through a pager like <code>more</code> or <code>less</code> to - avoid this. - </p> - - <p>Here's what happened when we ran this command:</p> - - <ol> - <li>The product client created an <code>XMLQuery</code> object - out of the string query <code>constant = pi</code>. - </li> - - <li>It asked the RMI Registry to tell it where (network - address) it could find the product service named - <code>MyProductService</code>. - </li> - - <li>After getting the response back from the RMI Registry, it - then contacted the product service over a network connection - (even if to the same local system) and asked it to handle - the query, passing the query object. - </li> - - <li>The product service had only one query handler, the - <code>ConstantHandler</code>, to which to delegate, so it - passed the XMLQuery to it. - </li> - - <li>The <code>ConstantHandler</code>'s <code>query</code> - method was called. It checked if the query was the kind it - wanted, extracted the desired mathematical constant's name, - checked for an acceptable requested MIME type, looked up the - constant's value, inserted it as a <code>Result</code> into - the <code>XMLQuery</code>, and returned the modified query. - </li> - - <li>The product service, seeing it had no other handlers to - try, returned the modified <code>XMLQuery</code> to the - product client over the network connection. - </li> - - <li>The product client took the first <code>Result</code> out - of the <code>XMLQuery</code>, called - <code>Result.getInputStream</code>, and copied each byte of - the result to the standard output. This wrote - <code>3.14159265...</code> to your window. - </li> - </ol> - - <p>If you change the <code>$PS_HOME/bin/pc</code> script again - so that instead of <code>-out</code> it's <code>-xml</code>, - you'll again see the XMLQuery as an XML document. The interesting part is the <code><queryResultSet></code>: - </p> - - <source><![CDATA[<queryResultSet> - <resultElement classified="false" validity="-1"> - <resultId>pi</resultId> - <resultMimeType>text/plain</resultMimeType> - <profId/> - <identifier/> - <resultHeader> - <headerElement> - <elemName>pi</elemName> - <elemType>real</elemType> - <elemUnit>none</elemUnit> - </headerElement> - </resultHeader> - <resultValue xml:space="preserve">3.14159265...</resultValue> - </resultElement> -</queryResultSet>]]></source> - - <p>I'll let you figure out how this maps to the - <code>Result</code> object we created in the code. OK, so is - this really interesting? Not really, except to note that the - actual result data, <code>3.1415265...</code>, appears in the - XML document. In the OODT framework, we call this a "small" - result, because the product data was embedded in the XMLQuery - object. Text data like <code>text/plain</code> products - appear just as text. Binary data like <code>image/jpeg</code> - and <code>application/octet-stream</code> get base-64 encoded - into text. - </p> - <p>As you can guess, there's a point at which encoded data goes - from being nice and small and tidy to just too large to - contain in a single object. In the OODT framework, we can use - a special query handler for such large products, but that's - another tutorial. - </p> - </section> - - <section name='Conclusion'> - <p>In this tutorial, we learned how to write a complete query - handler for a product server, including handling of postfix - boolean "where" stacks, lists of acceptable MIME types, and - result headers. We compiled the query handler, put it into a - jar file, and specified it to our product server. And we - could even query it and get data back. - </p> - - <p>Don't toss out this product server yet, though. Another - tutorial will use it and cover the infamous - <code>LargeProductQueryHandler</code>. - </p> - </section> - </body> -</document> http://git-wip-us.apache.org/repos/asf/oodt/blob/098cc4fa/product/src/test/org/apache/oodt/product/handlers/ofsn/util/OFSNUtilsTest.java ---------------------------------------------------------------------- diff --git a/product/src/test/org/apache/oodt/product/handlers/ofsn/util/OFSNUtilsTest.java b/product/src/test/org/apache/oodt/product/handlers/ofsn/util/OFSNUtilsTest.java deleted file mode 100644 index 31a4494..0000000 --- a/product/src/test/org/apache/oodt/product/handlers/ofsn/util/OFSNUtilsTest.java +++ /dev/null @@ -1,59 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one or more - * contributor license agreements. See the NOTICE file distributed with - * this work for additional information regarding copyright ownership. - * The ASF licenses this file to You under the Apache License, Version 2.0 - * (the "License"); you may not use this file except in compliance with - * the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - - -package org.apache.oodt.product.handlers.ofsn.util; - -import junit.framework.TestCase; -/* - * Licensed to the Apache Software Foundation (ASF) under one or more - * contributor license agreements. See the NOTICE file distributed with - * this work for additional information regarding copyright ownership. - * The ASF licenses this file to You under the Apache License, Version 2.0 - * (the "License"); you may not use this file except in compliance with - * the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -import java.util.Collections; - -/** - * Unit test for {@link OFSNUtils}. - * - * @author riverma - */ -public class OFSNUtilsTest extends TestCase { - public OFSNUtilsTest(String id) { - super(id); - } - - public void testValidateOFSN() { - - assertTrue(OFSNUtils.validateOFSN("/dataset/dir1")); - assertTrue(OFSNUtils.validateOFSN("/dataset/dir1/")); - assertTrue(OFSNUtils.validateOFSN("/dataset/dir1/file1.h5")); - assertFalse(OFSNUtils.validateOFSN("/dataset/../../../../../../etc/passwd")); - - } -} http://git-wip-us.apache.org/repos/asf/oodt/blob/098cc4fa/product/src/test/org/apache/oodt/xmlquery/ChunkedProductInputStreamTest.java ---------------------------------------------------------------------- diff --git a/product/src/test/org/apache/oodt/xmlquery/ChunkedProductInputStreamTest.java b/product/src/test/org/apache/oodt/xmlquery/ChunkedProductInputStreamTest.java deleted file mode 100644 index 9ab531a..0000000 --- a/product/src/test/org/apache/oodt/xmlquery/ChunkedProductInputStreamTest.java +++ /dev/null @@ -1,170 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one or more - * contributor license agreements. See the NOTICE file distributed with - * this work for additional information regarding copyright ownership. - * The ASF licenses this file to You under the Apache License, Version 2.0 - * (the "License"); you may not use this file except in compliance with - * the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - - -package org.apache.oodt.xmlquery; - -import java.io.IOException; -import java.util.Arrays; -import org.apache.oodt.product.ProductException; -import org.apache.oodt.product.Retriever; -import junit.framework.TestCase; -import java.io.ByteArrayOutputStream; - -/** - * Unit test for <code>ChunkedProductInputStream</code>. - * - * @author Kelly - * @version $Revision: 1.5 $ - */ -public class ChunkedProductInputStreamTest extends TestCase implements Retriever { - /** - * Creates a new <code>ChunkedProductInputStreamTest</code> instance. - * - * @param id Case name. - */ - public ChunkedProductInputStreamTest(String id) { - super(id); - } - - public void setUp() throws Exception { - super.setUp(); - data = new byte[4096]; - for (int i = 0; i < 4096; ++i) - data[i] = (byte) (i % 256); - } - - /** - * Test reading a single byte at a time. - * - * @throws IOException if an error occurs. - */ - public void testByteReading() throws IOException { - ChunkedProductInputStream in = new ChunkedProductInputStream("test", this, 4096); - for (int i = 0; i < 4096; ++i) - assertEquals(toByte(i % 256), toByte(in.read() & 0xff)); - assertEquals(-1, in.read()); - in.close(); - } - - public void testArrayReading() throws IOException { - ChunkedProductInputStream in = new ChunkedProductInputStream("test", this, 4096); - ByteArrayOutputStream out = new ByteArrayOutputStream(4096); - byte[] buf = new byte[256]; - int num; - while ((num = in.read(buf)) != -1) - out.write(buf, 0, num); - in.close(); - out.close(); - assertTrue(Arrays.equals(data, out.toByteArray())); - } - - /** - * Test reading and skipping by various amounts. - * - * @throws IOException if an error occurs. - */ - public void testReadingAndSkipping() throws IOException { - ChunkedProductInputStream in = new ChunkedProductInputStream("test", this, 4096); - - byte[] buf = new byte[4]; // Byte number: - assertEquals(0, in.read()); // 0 - assertEquals(0, in.skip(0)); // 0 - assertEquals(4, in.read(buf)); // 1, 2, 3, 4 - assertEquals(toByte(1), buf[0]); - assertEquals(toByte(2), buf[1]); - assertEquals(toByte(3), buf[2]); - assertEquals(toByte(4), buf[3]); - assertEquals(toByte(5), toByte(in.read())); // 5 - assertEquals(4, in.skip(4)); // 6, 7, 8, 9 - assertEquals(toByte(10), toByte(in.read())); // 10 - assertEquals(1000, in.skip(1000)); // 11, 12, ..., 1010 - assertEquals(toByte(1011 % 256), toByte(in.read())); // 1011 - - buf = new byte[1000]; - int toRead = 1000; - int index = 0; - while (toRead > 0) { // 1012, 1013, ..., 2011 - int numRead = in.read(buf, index, toRead); - if (numRead == -1) - fail("Premature EOF"); - toRead -= numRead; - index += numRead; - } - for (int i = 0; i < buf.length; ++i) - assertEquals(data[i + 1012], buf[i]); - - assertEquals(2, in.read(buf, 1, 2)); // 2012, 2013 - assertEquals(toByte(1012 % 256), buf[0]); - assertEquals(toByte(2012 % 256), buf[1]); - assertEquals(toByte(2013 % 256), buf[2]); - assertEquals(toByte(1015 % 256), buf[3]); - - assertEquals(2082, in.skip(2083)); // 2014, 2015, ..., 4095 - // Shouldn't we get the -1 read first, and THEN get an IOException on subsequent reads? - try { - assertEquals(-1, in.read()); - } catch (IOException ignore) {} - in.close(); - } - - /** - * Test reading into larger and larger arrays. - * - * @throws IOException if an error occurs. - */ - public void testWideningWindows() throws IOException { - // Scary; this test hangs on Windows. We really should investigate why as - // it could bite us on the bum in the future. - if (System.getProperty("os.name", "unknown").indexOf("Windows") != -1) return; - - byte[] read = new byte[4096]; - for (int size = 1; size <= 4096; size *= 2) { - byte[] buf = new byte[size]; - ChunkedProductInputStream in = new ChunkedProductInputStream("test", this, 4096); - ByteArrayOutputStream out = new ByteArrayOutputStream(4096); - int num; - while ((num = in.read(buf)) != -1) - out.write(buf, 0, num); - in.close(); - out.close(); - assertTrue(Arrays.equals(data, out.toByteArray())); - } - } - - public byte[] retrieveChunk(String id, long offset, int length) { - if (!id.equals("test")) - throw new IllegalArgumentException("Unknown id " + id); - if (offset < 0 || offset > 4096 || length < 0 || - (offset + length) > 4096 || (offset + length) < 0) - throw new IllegalArgumentException("Bad offset and/or length"); - - int index = (int) offset; - byte[] sub = new byte[length]; - System.arraycopy(data, index, sub, 0, length); - return sub; - } - - private static byte toByte(int b) { - return (byte) (b & 0xff); - } - - public void close(String id) {} - - /** Test data. */ - private byte[] data; -} http://git-wip-us.apache.org/repos/asf/oodt/blob/098cc4fa/product/src/test/org/apache/oodt/xmlquery/LargeResultTest.java ---------------------------------------------------------------------- diff --git a/product/src/test/org/apache/oodt/xmlquery/LargeResultTest.java b/product/src/test/org/apache/oodt/xmlquery/LargeResultTest.java deleted file mode 100644 index ff3679b..0000000 --- a/product/src/test/org/apache/oodt/xmlquery/LargeResultTest.java +++ /dev/null @@ -1,54 +0,0 @@ -/* - * Licensed to the Apache Software Foundation (ASF) under one or more - * contributor license agreements. See the NOTICE file distributed with - * this work for additional information regarding copyright ownership. - * The ASF licenses this file to You under the Apache License, Version 2.0 - * (the "License"); you may not use this file except in compliance with - * the License. You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - - -package org.apache.oodt.xmlquery; - -import junit.framework.TestCase; -import java.util.Collections; - -/** - * Unit test for {@link LargeResult}. - * - * @author Kelly - * @version $Revision: 1.2 $ - */ -public class LargeResultTest extends TestCase { - public LargeResultTest(String id) { - super(id); - } - - public void testLargeResults() { - LargeResult lr1 = new LargeResult("1.2.3", "text/plain", "JPL.Profile", "JPL.Resource", Collections.EMPTY_LIST, 1); - assertEquals("1.2.3", lr1.getID()); - assertEquals("text/plain", lr1.getMimeType()); - assertEquals("JPL.Profile", lr1.getProfileID()); - assertEquals("JPL.Resource", lr1.getResourceID()); - assertEquals(1, lr1.getSize()); - assertTrue(lr1.getHeaders().isEmpty()); - - Result r = new Result("2.3.4", "application/vnd.jpl.large-product", "JPL.Profile", "JPL.Resource", - Collections.EMPTY_LIST, "text/plain 2"); - LargeResult lr2 = new LargeResult(r); - assertEquals("text/plain", lr2.getMimeType()); - assertEquals(2, lr2.getSize()); - - LargeResult lr3 = new LargeResult(lr2); - assertEquals("text/plain", lr2.getMimeType()); - assertEquals(2, lr3.getSize()); - } -}
