typos and inconsistencies fixed

1 files changed, 64 insertions, 22 deletions

diff --git a/loael.tex b/loael.tex
index b82a370..6b9a2fb 100644
--- a/loael.tex
+++ b/loael.tex
@@ -24,7 +24,7 @@
 \usepackage[unicode=true]{hyperref}
 \PassOptionsToPackage{usenames,dvipsnames}{color} % color is loaded by hyperref
 \hypersetup{
-            pdftitle={Modeling Chronic Toxicity: A comparison of experimental variability with read across predictions},
+            pdftitle={Modeling Chronic Toxicity: A comparison of experimental variability with (Q)SAR/read-across predictions},
             pdfauthor={Christoph Helma1; David Vorgrimmler1; Denis Gebele1; Martin Gütlein2; Benoit Schilter3; Elena Lo Piparo3},
             pdfkeywords={(Q)SAR, read-across, LOAEL, experimental variability},
             colorlinks=true,
@@ -92,7 +92,7 @@
 \newcommand*\listoflistings{\listof{codelisting}{List of Listings}}
 
 \title{Modeling Chronic Toxicity: A comparison of experimental variability with
-read across predictions}
+(Q)SAR/read-across predictions}
 \author{Christoph Helma\textsuperscript{1} \and David Vorgrimmler\textsuperscript{1} \and Denis Gebele\textsuperscript{1} \and Martin Gütlein\textsuperscript{2} \and Benoit Schilter\textsuperscript{3} \and Elena Lo Piparo\textsuperscript{3}}
 \date{\today}
 
@@ -102,9 +102,9 @@ read across predictions}
 This study compares the accuracy of (Q)SAR/read-across predictions with
 the experimental variability of chronic LOAEL values from \emph{in vivo}
 experiments. We could demonstrate that predictions of the \texttt{lazar}
-lazar algrorithm within the applicability domain of the training data
-have the same variability as the experimental training data. Predictions
-with a lower similarity threshold (i.e.~a larger distance from the
+algrorithm within the applicability domain of the training data have the
+same variability as the experimental training data. Predictions with a
+lower similarity threshold (i.e.~a larger distance from the
 applicability domain) are also significantly better than random
 guessing, but the errors to be expected are higher and a manual
 inspection of prediction results is highly recommended.
@@ -150,8 +150,8 @@ et al. 2014). Since most of the time chemical food safety deals with
 life-long exposures to relatively low levels of chemicals, and because
 long-term toxicity studies are often the most sensitive in food
 toxicology databases, predicting chronic toxicity is of prime
-importance. Up to now, read across and quantitative structure-activity
-relationship (QSAR) have been the most used \emph{in silico} approaches
+importance. Up to now, read-across and Quantitative Structure Activity
+Relationships (QSAR) have been the most used \emph{in silico} approaches
 to obtain quantitative predictions of chronic toxicity.
 
 The quality and reproducibility of (Q)SAR and read-across predictions
@@ -167,7 +167,7 @@ overfitted models with little practical relevance.
 
 In the present study, automatic read-across like models were built to
 generate quantitative predictions of long-term toxicity. Two databases
-compiling chronic oral rat lowest adverse effect levels (LOAEL) as
+compiling chronic oral rat Lowest Adverse Effect Levels (LOAEL) as
 endpoint were used. An early review of the databases revealed that many
 chemicals had at least two independent studies/LOAELs. These studies
 were exploited to generate information on the reproducibility of chronic
@@ -255,7 +255,7 @@ numbers). Unique smiles from the OpenBabel library (OBoyle et al. 2011)
 were used for the identification of duplicated structures.
 
 Studies with undefined or empty LOAEL entries were removed from the
-databases LOAEL values were converted to mmol/kg\_bw/day units and
+databases. LOAEL values were converted to mmol/kg bw/day units and
 rounded to five significant digits. For prediction, validation and
 visualisation purposes -log10 transformations are used.
 
@@ -316,7 +316,8 @@ classified as a \emph{k-nearest-neighbor} algorithm.
 
 Apart from this basic workflow lazar is completely modular and allows
 the researcher to use any algorithm for similarity searches and local
-QSAR modelling. Within this study we are using the following algorithms:
+QSAR modelling. Algorithms used within this study are described in the
+following sections.
 
 \subsubsection{Neighbor identification}\label{neighbor-identification}
 
@@ -333,7 +334,7 @@ chemical environment using the atom types of connected atoms.
 MolPrint2D fingerprints are generated dynamically from chemical
 structures and do not rely on predefined lists of fragments (such as
 OpenBabel FP3, FP4 or MACCs fingerprints or lists of
-toxocophores/toxicophobes). This has the advantage the they may capture
+toxocophores/toxicophobes). This has the advantage that they may capture
 substructures of toxicological relevance that are not included in other
 fingerprints. Unpublished experiments have shown that predictions with
 MolPrint2D fingerprints are indeed more accurate than other OpenBabel
@@ -357,11 +358,18 @@ threshold) and the number of predictable compounds (low threshold). As
 it is in many practical cases desirable to make predictions even in the
 absence of closely related neighbors, we follow a tiered approach:
 
-First a similarity threshold of 0.5 is used to collect neighbors, to
-create a local QSAR model and to make a prediction for the query
-compound. If any of this steps fail, the procedure is repeated with a
-similarity threshold of 0.2 and the prediction is flagged with a warning
-that it might be out of the applicability domain of the training data.
+\begin{itemize}
+\tightlist
+\item
+  First a similarity threshold of 0.5 is used to collect neighbors, to
+  create a local QSAR model and to make a prediction for the query
+  compound.
+\item
+  If any of these steps fails, the procedure is repeated with a
+  similarity threshold of 0.2 and the prediction is flagged with a
+  warning that it might be out of the applicability domain of the
+  training data.
+\end{itemize}
 
 Compounds with the same structure as the query structure are
 automatically
@@ -413,7 +421,7 @@ domain. Quantitative applicability domain information can be obtained
 from the similarities of individual neighbors.
 
 Local regression models consider neighbor similarities to the query
-compound, by weighting the contribution of each neighbor is by its
+compound, by weighting the contribution of each neighbor is by
 similarity. The variability of local model predictions is reflected in
 the 95\% prediction interval associated with each prediction.
 
@@ -471,7 +479,7 @@ baseline for evaluating prediction performance.
 In order to compare the structural diversity of both databases we
 evaluated the frequency of functional groups from the OpenBabel FP4
 fingerprint. Figure~\ref{fig:fg} shows the frequency of functional
-groups in both databases 139 functional groups with a frequency
+groups in both databases. 139 functional groups with a frequency
 \textgreater{} 25 are depicted, the complete table for all functional
 groups can be found in the supplemental material at
 \href{https://github.com/opentox/loael-paper/blob/submission/data/functional-groups.csv}{GitHub}.
@@ -574,15 +582,15 @@ analysis.}\label{fig:datacorr}
 \subsubsection{Local QSAR models}\label{local-qsar-models}
 
 In order to compare the performance of \emph{in silico} read across
-models with experimental variability we are using compounds with
-multiple measurements as a test set (375 measurements, 155 compounds).
+models with experimental variability we used compounds with multiple
+measurements as a test set (375 measurements, 155 compounds).
 \texttt{lazar} read across predictions were obtained for 155 compounds,
 37 predictions failed, because no similar compounds were found in the
 training data (i.e.~they were not covered by the applicability domain of
 the training data).
 
-Experimental data and 95\% prediction intervals overlapped in 100\% of
-the test examples.
+In 100\% of the test examples experimental LOAEL values were located
+within the 95\% prediction intervals.
 
 Figure~\ref{fig:comp} shows a comparison of predicted with experimental
 values. Most predicted values were located within the experimental
@@ -787,6 +795,40 @@ since evidence suggest that exposure duration has little impact on the
 levels of NOAELs/LOAELs (Zarn, Engeli, and Schlatter 2011, Zarn, Engeli,
 and Schlatter (2013)).
 
+\subsubsection{\texorpdfstring{\texttt{lazar}
+predictions}{lazar predictions}}\label{lazar-predictions}
+
+Table~\ref{tbl:common-pred}, Table~\ref{tbl:cv}, Figure~\ref{fig:comp},
+Figure~\ref{fig:corr} and Figure~\ref{fig:cv} clearly indicate that
+\texttt{lazar} generates reliable predictions for compounds within the
+applicability domain of the training data (i.e.~predictions without
+warnings, which indicates a sufficient number of neighbors with
+similarity \textgreater{} 0.5 to create local random forest models).
+Correlation analysis (Table~\ref{tbl:common-pred}, Table~\ref{tbl:cv})
+shows, that errors (\(RMSE\)) and explained variance (\(r^2\)) are
+comparable to experimental variability of the training data.
+
+Predictions with a warning (neighbor similarity \textless{} 0.5 and
+\textgreater{} 0.2 or weighted average predictions) are a grey zone.
+They still show a strong correlation with experimental data, but the
+errors are larger than for compounds within the applicability domain
+(Table~\ref{tbl:common-pred}, Table~\ref{tbl:cv}). Expected errors are
+displayed as 95\% prediction intervals, which covers 100\% of the
+experimental data. The main advantage of lowering the similarity
+threshold is that it allows to predict a much larger number of
+substances than with more rigorous applicability domain criteria. As
+each of this prediction could be problematic, they are flagged with a
+warning to alert risk assessors that further inspection is required.
+This can be done in the graphical interface
+(\url{https://lazar.in-silico.ch}) which provides intuitive means of
+inspecting the rationales and data used for read across predictions.
+
+Finally there is a substantial number of compounds (37), where no
+predictions can be made, because there are no similar compounds in the
+training data. These compounds clearly fall beyond the applicability
+domain of the training dataset and in such cases it is preferable to
+avoid predictions instead of random guessing. --\textgreater{}
+
 TODO: GUI screenshot
 
 \section{Summary}\label{summary}