xref: /aosp_15_r20/external/apache-xml/test/java/xdocs/sources/tests/faq.xml (revision 1212f9a0ffdc28482b8821715d2222bf16dc14e2)
1 <?xml version="1.0" standalone="no"?>
2 <!DOCTYPE s1 SYSTEM "sbk:/style/dtd/document.dtd">
3 <!--
4  * Licensed to the Apache Software Foundation (ASF) under one
5  * or more contributor license agreements. See the NOTICE file
6  * distributed with this work for additional information
7  * regarding copyright ownership. The ASF licenses this file
8  * to you under the Apache License, Version 2.0 (the  "License");
9  * you may not use this file except in compliance with the License.
10  * You may obtain a copy of the License at
11  *
12  *     http://www.apache.org/licenses/LICENSE-2.0
13  *
14  * Unless required by applicable law or agreed to in writing, software
15  * distributed under the License is distributed on an "AS IS" BASIS,
16  * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
17  * See the License for the specific language governing permissions and
18  * limitations under the License.
19 -->
20 <s1 title="Frequently asked questions">
21 
22   <s2 title="In progress">
23     <p>We need to start a FAQ for testing Xalan, unfortunately the faq doctype
24     doesn't properly work in this tree, so I'm using an s1/s2 doc instead temporarily.
25     Some topics include:</p>
26 <ul>
27 <li><link anchor="viewresults">Pretty-printing results</link></li>
28 <li><link anchor="debug">Run tests in a debugger, etc.</link></li>
29 </ul>
30   </s2>
31   <s2 title="Prepare to run tests">
32     <p>CVSROOT=:pserver:<em>user</em>@cvs.apache.org:/home/cvs etc.<br/>
33     or CVSROOT=:pserver:anoncvs@cvs.apache.org:/home/cvspublic etc.<br/>
34     cvs checkout xml-xalan/java<br/>
35     cvs checkout xml-xalan/test<br/>
36     cd xml-xalan/java<br/>
37     build jar<br/>
38     cd ../test<br/>
39     build jar<br/>
40     build -projecthelp  - to get a list of targets<br/>
41     </p>
42   </s2>
43 
44   <s2 title="Run conformance tests">
45     <p>The conf set of tests can be run in a variety of ways, especially with each different flavor.
46     For a list of flavors, see xml-xalan/test/java/src/org/apache/qetest/xslwrapper/TransformWrapperFactory.properties</p>
47     <ul>
48     <li><source>build conf</source>  (default: trax.systemId)<br/><br/></li>
49     <li><source>build conf -Dconf.flavor=trax.sax</source>   (uses SAX IMPORTANT!)<br/><br/></li>
50     <li><source>build conf -Dconf.flavor=trax.dom</source>   (uses dom IMPORTANT!)<br/><br/></li>
51     <li>trax.file - File object instead of systemId<br/><br/></li>
52     <li>trax.stream - InputStreams - IMPORTANT!<br/><br/></li>
53     <li>trax.localPath - uses local paths instead of URLs, experimental<br/><br/></li>
54     <li>trax.systemId3 - does systemId transform three times in a row, experimental<br/><br/></li>
55     <li>process - uses command line class - IMPORTANT!<br/><br/></li>
56     </ul>
57   </s2>
58 
59   <s2 title="Run contrib/perf tests">
60     <p>Same as running conf tests, except substitute 'contrib' for 'conf' everywhere,
61     or 'perf' instead.  Note that 'perf' uses a custom Testlet testing algorithim
62     that iterates several times and outputs custom perf elements.</p>
63     <ul>
64     <li><source>build contrib -Dcontrib.flavor=trax.sax</source><br/><br/></li>
65     <li><source>build perf -Dperf.flavor=trax.stream</source><br/><br/></li>
66     </ul>
67   </s2>
68 
69   <anchor name="xsltc"/>
70   <s2 title="Run XSLTC-related tests">
71     <p>Note that currently some of the XSLTC-specific tests are
72     built with separate targets; in the future as the XSLTC engine
73     is more completely integrated into the Xalan core, we hope to
74     merge the tests as well.</p>
75     <p>Note that many of the tests that use TrAX/JAXP should work
76     equally well with XSLTC since they have their own version of
77     a TransformerFactory; remember to set the appropriate system
78     property or classpath appropriately.  The various org.apache.qetest.trax
79     API tests should also work normally with XSLTC.</p>
80   </s2>
81 
82   <anchor name="xsltmark"/>
83   <s2 title="Run XSLTMARK perf tests">
84     <p>Xalan has it's own more detailed automation harness for running
85     stylesheet tests and capturing performance metrics.  You can run the XSLTMARK
86     set of stylesheets using Xalan's harness to see more detailed performance
87     data.</p>
88     <ul>
89     <li>Copy xsltmark/testcases to xml-xalan/test (so it's xml-xalan/test/testcases)<br/><br/></li>
90     <li>Copy xsltmark/testcases/default.conf to xml-xalan/test/xsltmark.filelist<br/><br/></li>
91     <li>cd xml-xalan/test<br/><br/></li>
92     <li>Search-and-replace xsltmark.filelist 't=' 't=testcases/'<br/><br/></li>
93     <li>Search-and-replace xsltmark.filelist 'e=' 'e=testcases/'<br/><br/></li>
94     <li><source>build perf -Dperf.fileList=xsltmark.filelist</source><br/><br/></li>
95     <li><source>java -classpath blah org.apache.xalan.xslt.Process -in results-perf/results.xml -xsl PerfScanner.xsl -out results-perf/PerfReport.html</source><br/><br/></li>
96     <li>(where blah includes xml-apis.jar;xalan.jar;xercesImpl.jar)<br/><br/></li>
97     </ul>
98   </s2>
99 
100   <anchor name="viewresults"/>
101   <s2 title="View results in HTML">
102     <p>Every test creates a results.xml (or TestName.xml) file of all the results
103     the test has performed - no need to examine the console, since all the info will
104     be here.  We have a couple of prototype stylesheets to view the results in a
105     semi-pretty HTML style.</p>
106     <p>There is also a tableResults.xsl stylesheet for creating a table of conformance
107       test results. For documentation on how to use the stylesheet, please consult the section
108       <link idref="run" anchor="how-to-view-results">How-to: View Results</link>.
109     </p>
110     <ul>
111     <li><source>build perf</source><br/>
112     (Creates results-perf/results.xml)<br/></li>
113     <li><source>java -classpath blah org.apache.xalan.xslt.Process -in results-perf/results.xml -xsl FailScanner.xsl -out results-perf/FailReport.html</source><br/><br/></li>
114     <li><source>java -classpath blah org.apache.xalan.xslt.Process -in results-perf/results.xml -xsl PerfScanner.xsl -out results-perf/PerfReport.html</source><br/><br/></li>
115     <li><source>build alltest</source><br/>
116     (Creates a <b>LOT</b> of results in results-alltest/**)<br/></li>
117     <li><source>build scan</source><br/>
118     (Runs ResultScanner by default on results-alltest/**)<br/></li>
119     <li><source>build scan -Dscan.outputDir=results-perf</source>  etc.<br/></li>
120     <li><source>java -classpath blah;testxsl.jar org.apache.qetest.xsl.ResultScanner results-alltest</source><br/>
121     (This uses ResultScanner.xsl to style <b>all</b> results in the whole tree under
122     results-alltest into a single ResultReport.html in the current directory; it currently
123     uses FailScanner.xsl to only include fail results)<br/></li>
124     </ul>
125   </s2>
126 
127   <s2 title="Include/Exclude tests">
128     <p>Run just a subset of tests, or exclude tests using simple command line options.</p>
129     <ul>
130     <li><source>build conf -Dconf.category=axes;boolean</source> - Only run those directories<br/><br/></li>
131     <li><source>build conf -Dconf.excludes=axes107.xsl;boolean12.xsl</source> - Skip those explicit xsl filenames<br/><br/></li>
132     </ul>
133   </s2>
134 
135   <anchor name="conf.one"/>
136   <s2 title="Run a single conf test">
137     <ul>
138     <li><source>build conf.one -Dconf.test=axes44 -Dconf.flavor=trax.stream</source><br/>
139     This will run just the axes44.xsl test, using the normal Testlet algorithim <b>and</b>
140     using whatever flavor you choose (which makes it easy to see if single tests
141     run properly using SAX, DOM, streams, whatever)<br/></li>
142     </ul>
143   </s2>
144 
145   <s2 title="Run all API tests">
146     <ul>
147     <li><source>build api -DtestClass=TransformerAPITest</source><br/></li>
148     <li>This runs all the available specific API tests, which includes all the
149     API tests in the smoketest, plus other tests that fail due to known bugzilla reports.<br/></li>
150     <li>Note that in the case of <source>testClass</source>, you do <b>not</b>
151     prefix the name of the option with conf,api,perf,etc.<br/><br/></li>
152     </ul>
153   </s2>
154 
155   <s2 title="Run a LOT tests">
156     <ul>
157     <li><source>build alltest</source> (runs all,alltest.other,alltest.conf,alltest.contrib)<br/></li>
158     <li><source>build alltest.other</source> runs all API tests, extensions, bugzilla, threading, and perf tests<br/></li>
159     <li><source>build alltest.conf</source> run all flavors of conf tests<br/></li>
160     <li><source>build alltest.contrib</source>  run all flavors of contrib tests<br/></li>
161     <li><source>build alltest.features</source> run all conf and contrib tests,
162     matrixed over all flavors, twice: once with the feature incremental set to true,
163     once with optimize set to false (note: not all features matter with all flavors,
164     but we run them all anyway.  Several meg of output!)<br/></li>
165     </ul>
166   </s2>
167 
168   <anchor name="debug"/>
169   <s2 title="Debug the tests">
170     <p>It's quite simple to run the tests standalone, without using Ant - this is
171     suitable for running in your debugger or what-not (or in environments where
172     you feel that the overhead of running Ant might interfere with the test).</p>
173     <p>Normally, we have the build.bat/.sh files and the Ant script build.xml
174     manage parsing the command line and setting up the CLASSPATH and options for
175     running tests.  However all tests can be run on the command line, and will
176     accept options in either a -load file.properties block, or on the command line itself.</p>
177     <ul>
178     <li><source>debugapi.bat org.apache.qetest.trax.TransformerAPITest</source><br/>
179     Runs just one API test without using Ant; edit debugapi.properties to
180     change parameters passed to the test.<br/></li>
181     <li><source>debugconf.bat [options]</source><br/>
182     Runs a set of conf/ stylesheet tests using the normal StylesheetTestletDriver;
183     see debugconf.properties for settable options.  Note that this method does not
184     directly support 'conf.one' to run only a <b>single</b> test.<br/></li>
185     <li>Setup everything manually.  Minimal setup is: CLASSPATH has
186     xalan.jar;serializer.jar,xercesImpl.jar;xml-apis.jar;testxsl.jar, command line parameters or
187     -load file.properties includes other test properties, like inputDir, goldDir,
188     outputDir, logFile, etc.</li>
189     </ul>
190   </s2>
191 </s1>
192