API-Definition 
===================================================================

see http://opentox.org/dev/apis/api-1.1/Validation


API-Examples Validation
===================================================================

how to
-------------------------------------------------------------------
the examples listed below can be performed with the command line tool curl 


get list of available validations
-------------------------------------------------------------------

>>> curl <validation_service>

result example (accept-header: application/rdf+xml)			
<<< not yet supported

result example (accept-header: text/uri-list)
<<< <validation_service>/<validation_id>
<<< <validation_service>/validation_id_i
<<< <validation_service>/validation_id_j


get validation
-------------------------------------------------------------------

>>> curl <validation_service>/<validation_id>

supported accept-headers: "text/x-yaml" and "application/rdf+xml"			
	
	
validate model on test-dateset
-------------------------------------------------------------------	

>>> curl -X POST -d model_uri="<model_uri>" \
 -d test_dataset_uri="<test_dataset_uri>" \
 -d test_target_dataset_uri="<dataset_uri>" \
 <validation_service>/test_set_validation
 
optional params:
test_target_dataset_uri, default is test_dataset_uri
	
result example (accept-header: application/rdf+xml)			
<<< not yet supported

result example (accept-header: text/uri-list)
<<< <validation_service>/id_i


validate an algorithm on a training- and test-dataset
-------------------------------------------------------------------

>>> curl -X POST -d algorithm_uri="<algorithm_uri>" \
 -d training_dataset_uri="<training_dataset_uri>" \
 -d test_dataset_uri="<test_dataset_uri>" \
 -d test_target_dataset_uri="<dataset_uri>" \
 -d prediction_feature="<prediction_feature>" \
 -d algorithm_params="<algorithm_params>" \
 <validation_service>/training_test_validation

optional params:
algorithm_params, default is empty
test_target_dataset_uri, default is test_dataset_uri
	
result example (accept-header: application/rdf+xml)			
<<< not yet supported

result example (accept-header: text/uri-list)
<<< <validation_service>/id_i

 
validate an algorithm on a dataset via training-test-split
-------------------------------------------------------------------

>>> curl -X POST -d algorithm_uri="<algorithm_uri>" \
 -d dataset_uri="<dataset_uri>" \
 -d prediction_feature="<prediction_feature>" \
 -d algorithm_params="<algorithm_params>" \
 -d split_ratio=0.9 \
 -d random_seed=2 \
 <validation_service>/training_test_split

optional params:
algorithm_params, default is empty
split_ratio, default is 0.67
random_seed, default is 1
	
result example (accept-header: application/rdf+xml)			
<<< not yet supported

result example (accept-header: text/uri-list)
<<< <validation_service>/id_i


validate an algorithm on a dataset via bootstrapping
-------------------------------------------------------------------

>>> curl -X POST -d algorithm_uri="<algorithm_uri>" \
 -d dataset_uri="<dataset_uri>" \
 -d prediction_feature="<prediction_feature>" \
 -d algorithm_params="<algorithm_params>" \
 -d random_seed=2 \
 <validation_service>/bootstrapping

optional params:
algorithm_params, default is empty
random_seed, default is 1
	
result example (accept-header: application/rdf+xml)			
<<< not yet supported

result example (accept-header: text/uri-list)
<<< <validation_service>/id_i


get list of available crossvalidations
-------------------------------------------------------------------

>>> curl <validation_service>/crossvalidation

result example (accept-header: application/rdf+xml)			
<<< not yet supported

result example (accept-header: text/uri-list)
<<< <validation_service>/crossvalidation/<crossvalidation_id>
<<< <validation_service>/crossvalidation/crossvalidation_id_i
<<< <validation_service>/crossvalidation/crossvalidation_id_j


get crossvalidation
-------------------------------------------------------------------

>>> curl <validation_service>/crossvalidation/<crossvalidation_id>

supported accept-headers: "text/x-yaml" and "application/rdf+xml"			
	

validate an algorithm on a dataset via crossvalidation
(may take a couple of minutes)
-------------------------------------------------------------------

>>> curl -X POST -d algorithm_uri="<algorithm_uri>" \
 -d dataset_uri="<dataset_uri>" \
 -d prediction_feature="<prediction_feature>" \
 -d algorithm_params="<algorithm_params>" \
 -d num_folds=5 \
 -d random_seed=2 \
 -d stratified=false \
 <validation_service>/crossvalidation

optional params:
algorithm_params, default is empty
num_folds, default is 10
random_seed, default is 1
stratified, default is false
	
result example (accept-header: application/rdf+xml)			
<<< not yet supported

result example (accept-header: text/uri-list)
<<< <validation_service>/crossvalidation/id_i



API-Examples Reports
===================================================================


get list of available report types
-------------------------------------------------------------------

>>> curl <validation_service>/report

result example (accept-header: application/rdf+xml)			
<<< not yet supported

result example (accept-header: text/uri-list)
<<< <validation_service>/report/validation
<<< <validation_service>/report/crossvalidation
<<< <validation_service>/report/algorithm_comparison


get list of available reports of type validation
-------------------------------------------------------------------

>>> curl <validation_service>/report/validation

result example (accept-header: application/rdf+xml)			
<<< not yet supported

result example (accept-header: text/uri-list)
<<< <validation_service>/report/validation/<validation_report_id>
<<< <validation_service>/report/validation/id_i
<<< <validation_service>/report/validation/id_j


get validation report
-------------------------------------------------------------------

>>> curl <validation_service>/report/validation/<validation_report_id>

Supported formats (accept-headers):
* "text/xml" content of report in docbook-article format
* "text/html" report formated with default docbook-article-xsl
* "text/x-yaml" returns report object with meta-info (without the actual report content)
* "application/rdf+xml" returns report object with meta-info (without the actual report content)

Hint: Visit <validation_service>/report/validation/<validation_report_id> with a browser to see the report in html format


create validation report from validation
-------------------------------------------------------------------

>>> curl -X POST -d validation_uris="<validation_service>/<validation_id>" \
 <validation_service>/report/validation

result example (accept-header: application/rdf+xml)			
<<< not yet supported

result example (accept-header: text/uri-list)
<<< <validation_service>/report/validation/id_i


get list of available reports of type crossvalidation
-------------------------------------------------------------------

>>> curl <validation_service>/report/crossvalidation

result example (accept-header: application/rdf+xml)			
<<< not yet supported

result example (accept-header: text/uri-list)
<<< <validation_service>/report/crossvalidation/<crossvalidation_report_id>
<<< <validation_service>/report/crossvalidation/id_i
<<< <validation_service>/report/crossvalidation/id_j


get crossvalidation report
-------------------------------------------------------------------

>>> curl <validation_service>/report/crossvalidation/<crossvalidation_report_id>

Supported formats (accept-headers):
* "text/xml" content of report in docbook-article format
* "text/html" report formated with default docbook-article-xsl
* "text/x-yaml" returns report object with meta-info (without the actual report content)
* "application/rdf+xml" returns report object with meta-info (without the actual report content)

Hint: Visit <validation_service>/report/crossvalidation/<crossvalidation_report_id> with a browser to see the report in html format


create crossvalidation report from crossvalidation
-------------------------------------------------------------------

>>> curl -X POST -d validation_uris="<validation_service>/crossvalidation/<crossvalidation_id>" \
 <validation_service>/report/crossvalidation

result example (accept-header: application/rdf+xml)			
<<< not yet supported

result example (accept-header: text/uri-list)
<<< <validation_service>/report/crossvalidation/id_i


manually format html report
-------------------------------------------------------------------

>>> curl -X POST -d css_style_sheet=<css_file> \
  <validation_service>/report/crossvalidation/<crossvalidation_report_id>/format_html 

This explicity invokes the report convertion to html, with css-style-sheet as parameter. 

result example (accept-header: application/rdf+xml)			
<<< not yet supported

result example (accept-header: text/uri-list)
<<< <validation_service>/report/crossvalidation/id_i



Filter resources
===================================================================

It is possible to filter the list of available resources (i.e. validations, crossvalidation, reports) according to property-values.


filter available validations for a particular training dataset and model
-------------------------------------------------------------------

>>> curl <validation_service>?trainingDataset=<training_dataset_uri>\&model=<model_uri>

alternatively, use property names in (internal) yaml format:

>>> curl <validation_service>?training_dataset_uri=<training_dataset_uri>\&model_uri=<model_uri>


search available validation report for a particular algorithm
-------------------------------------------------------------------

>>> curl <validation_service>/report/validation?algorithm=lazar

alternatively, use property names in (internal) yaml format:

>>> curl <validation_service>/report/validation?algorithm_uris=lazar



Reach reporting
===================================================================


get list of available qmrf report
-------------------------------------------------------------------

>>> curl <validation_service>/reach_report/qmrf

result example (accept-header: application/rdf+xml)			
<<< not yet supported

result example (accept-header: text/uri-list)
<<< <validation_service>/reach_report/qmrf/<qmrf_id>
<<< <validation_service>/reach_report/qmrf/report_id_i
<<< <validation_service>/reach_report/qmrf/report_id_j


create a QMRF from a model
-------------------------------------------------------------------	

>>> curl -X POST -d model_uri="<model_uri>" \
 <validation_service>/reach_report/qmrf
 
result example (accept-header: application/rdf+xml)			
<<< not yet supported

result example (accept-header: text/uri-list)
<<< <validation_service>/reach_report/qmrf/id