JAFROC SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

================================================================================

JAFROC Version 4.2.1

Last Compilation Dates
	JAFROC_CORE.DLL : Aug  7 2014 15:10:56
	JAFROC.EXE      : Aug  7 2014 15:11:24

	Today's date: 5/15/2020 12:55:53



FOM selected         :     
Input  Data Filename :     \\vmware-host\Shared Folders\VmWareShared\VanDyke.xlsx
Output Data Filename :     \\vmware-host\Shared Folders\VmWareShared\VanDyke_ROC.txt

================================================================================
 Number of Readers        :  5
 Number of Treatments     :  2
 Number of Normal Cases   :  69
 Number of Abnormal Cases :  45
 fraction normal cases    :  0.605
 min    LESIONS_PER_IMAGE :  1
 max    LESIONS_PER_IMAGE :  1
 mean   LESIONS_PER_IMAGE :  1
 Total  LESIONS           :  45
 Inc. Loc. Frac.          :  0.000

================================================================================

For TRT = 0         , and RDR = 0         , max FPF =  1.000.
For TRT = 0         , and RDR = 1         , max FPF =  1.000.
For TRT = 0         , and RDR = 2         , max FPF =  1.000.
For TRT = 0         , and RDR = 3         , max FPF =  1.000.
For TRT = 0         , and RDR = 4         , max FPF =  1.000.
For TRT = 1         , and RDR = 0         , max FPF =  1.000.
For TRT = 1         , and RDR = 1         , max FPF =  1.000.
For TRT = 1         , and RDR = 2         , max FPF =  1.000.
For TRT = 1         , and RDR = 3         , max FPF =  1.000.
For TRT = 1         , and RDR = 4         , max FPF =  1.000.

================================================================================

 Avg. number of non-lesion localization marks per reader on normal images:  1.000.
 Avg. number of non-lesion localization marks per reader on abnormal images:  0.000.
 Avg. number of lesion localization marks per reader :  1.000.

================================================================================

 ====================================================================
 *****                        Overview                          *****
 ====================================================================
 Three analyses are presented: 
 (1) Analysis 1 treats both readers and cases as random samples
     --results apply to the reader and case populations;
 (2) Analysis 2 treats only cases as a random sample
     --results apply to the population of cases but only for the
     readers used in this study; and
 (3) Analysis 3 treats only readers as a random sample
     --results apply to the population of readers but only for the
     cases used in this study.
 
 For all three analyses, the null hypothesis of equal treatments is
 tested in part (a), treatment difference 95% confidence intervals
 are given in part (b), and treatment 95% confidence intervals are
 given in part (c).  Parts (a) and (b) are based on the treatment x
 reader x case ANOVA while part (c) is based on the reader x case
 ANOVA for the specified treatment; these ANOVA tables are displayed
 before the analyses.  Different error terms are used as indicated
 for parts (a), (b), and (c) according to whether readers and cases
 are treated as fixed or random factors.  Note that the treatment
 confidence intervals in part (c) are based only on the data for the
 specified treatment, rather than the pooled data.  Treatment
 difference 95% confidence intervals for each reader are presented
 in part (d) of Analysis 2; each interval is based on the treatment
 x case ANOVA table (not included) for the specified reader.
 ===========================================================================
 *****                            Estimates                            *****
 ===========================================================================

                        TREATMENT
              -----------------------
  READER      0            1         
----------    ----------   ----------
0             0.91964573   0.94782609
1             0.85877615   0.90531403
2             0.90386474   0.92173910
3             0.97310787   0.99935585
4             0.82979065   0.92995167
 
 
 TREATMENT MEANS (averaged across readers)
----------    -----------------------------
0             0.89703703
1             0.94083726
 
 

 TREATMENT MEAN DIFFERENCES
----------   ----------    -----------
0          - 1             -0.04380023
 
 
 
 
 ===========================================================================
 *****                          ANOVA Tables                           *****
 ===========================================================================
 
 TREATMENT X READER X CASE ANOVA
 
Source            SS               DF             MS        
------   --------------------    ------   ------------------
     T             0.54676316         1           0.54676316
     R             1.74930715         4           0.43732679
     C            44.84618708       113           0.39686891
    TR             0.25127013         4           0.06281753
    TC            11.28283814       113           0.09984813
    RC            29.15444801       452           0.06450099
   TRC            18.06719311       452           0.03997167
 TOTAL           105.89800676      1139
 
 
 
 READER X CASE ANOVAs for each treatment
 
 
                        Mean Squares
 Source     df   0            1         
 ------    ---   ----------   ----------   
      R      4   0.35141971   0.14872460
      C    113   0.33629383   0.16042320
     RC    452   0.06043604   0.04403661 
 
 ===========================================================================
 *****                  Variance components estimates                  *****
 ===========================================================================
 
 DBM Variance Component Estimates
 (for sample size estimation for future studies)
 Note: These are unbiased ANOVA estimates which can be negative
 
     DBM Component            Estimate    
 -----------------------  ----------------
 Var(R)                         0.00153500
 Var(C)                         0.02724915
 Var(T*R)                       0.00020040
 Var(T*C)                       0.01197529
 Var(R*C)                       0.01226466
 Var(T*R*C) + Var(Error)        0.03997167
 
 Obuchowski-Rockette variance component and covariance estimates
 (for sample size estimation for future studies)
 Note: These are unbiased ANOVA estimates which can be negative
 
     OR Component             Estimate    
 -----------------------  ----------------
 Reader                         0.00153500
 Treatment*Reader               0.00020040
 COV1                           0.00034661
 COV2                           0.00034407
 COV3                           0.00023903
 ERROR                          0.00080229
 
 
 ===========================================================================
 *****           Analysis 1: Random Readers and Random Cases           *****
 ===========================================================================
 (Results apply to the population of readers and cases)


    a) Test for H0: Treatments have the same qT
 
 Source        DF    Mean Square      F value  Pr > F 
 ----------  ------  ---------------  -------  -------
 Treatment        1       0.54676316     4.46   0.0517
 Error        15.26       0.12269399
 Error term: MS(TR) + max[MS(TC)-MS(TRC),0]
 
 Conclusion: The treatment q8s are not significantly different,
             F(1,15.26) = 4.46, p = 0.0517.


    b) 95% confidence intervals for treatment differences
 
       Treatment         Estimate   StdErr      DF      t     Pr > t          95% CI      
----------   ----------  --------  --------  -------  ------  -------  -------------------
0          - 1           -0.04380   0.02075    15.26   -2.11   0.0517  -0.08796 ,  0.00036
 
 H0: the two treatments are equal.
 Error term: MS(TR) + max[MS(TC)-MS(TRC),0]
 

    c) 95% treatment confidence intervals based on reader x case ANOVAs
       for each treatment (each analysis is based only on data for the
       specified treatment
 
  Treatment     Area      Std Error     DF     95% Confidence Interval 
  ----------  ----------  ----------  -------  -------------------------
  0           0.89703703  0.03317359    12.74  (0.82522361 , 0.96885046)
  1           0.94083734  0.02156635    12.71  (0.89413784 , 0.98753684)
 Error term: MS(R) + max[MS(C)-MS(RC),0]
 
 
 
 ===========================================================================
 *****           Analysis 2: Fixed Readers and Random Cases            *****
 ===========================================================================
 
 (Results apply to the population of cases but only for the readers
 used in this study)
 
    a) Test for H0: Treatments have the same 
 
 Source        DF    Mean Square      F value  Pr > F 
 ----------  ------  ---------------  -------  -------
 Treatment        1       0.54676316     5.48   0.0210
 Error          113       0.09984813
 Error term: MS(TC)
 
 Conclusion: The treatment s are not equal,
             F(1,113.00) = 5.48, p = 0.0210.


    b) 95% confidence intervals for treatment differences
 
       Treatment         Estimate   StdErr      DF      t     Pr > t          95% CI      
----------   ----------  --------  --------  -------  ------  -------  -------------------
0          - 1           -0.04380   0.01872      113   -2.34   0.0210  -0.08088 , -0.00672
 
 H0: the two treatments are equal.
 Error term: MS(TC)
 

    c) 95% treatment confidence intervals based on reader x case ANOVAs
       for each treatment (each analysis is based only on data for the
       specified treatment
 
  Treatment     Area      Std Error     DF     95% Confidence Interval 
  ----------  ----------  ----------  -------  -------------------------
  0           0.89703703  0.02428969      113  (0.84891477 , 0.94515930)
  1           0.94083734  0.01677630      113  (0.90760047 , 0.97407422)
 Error term: MS(C)
 
 
 
 TREATMENT X CASE ANOVAs for each reader
 

                        Sum of Squares
 Source     df   0             1             2             3             4             
 ------    ---   -----------   -----------   -----------   -----------   -----------   
      T      1     0.0452655     0.1234490     0.0182111     0.0392705     0.5718372   
      C    113    13.8142077    24.0404653    15.4282394     1.9537097    18.7640131   
     TC    113     4.1922524     4.4557771     6.2738149     1.9257950    12.5023919   
 

                        Mean Squares
 Source     df   0             1             2             3             4             
 ------    ---   -----------   -----------   -----------   -----------   -----------   
      T      1     0.0452655     0.1234490     0.0182111     0.0392705     0.5718372   
      C    113     0.1222496     0.2127475     0.1365331     0.0172895     0.1660532   
     TC    113     0.0370996     0.0394317     0.0555205     0.0170424     0.1106406   
 
 


    d) Treatment-by-case ANOVA CIs for each reader 
       (each analysis is based only on data for the specified reader)
 
  Reader         Treatment        Estimate  StdErr       DF      t     Pr > t          95% CI      
---------- ---------- ----------  --------  --------  -------  ------  -------  -------------------
0          0         -1           -0.02818   0.02551      113   -1.10   0.2717  -0.07872 ,  0.02236
1          0         -1           -0.04654   0.02630      113   -1.77   0.0795  -0.09865 ,  0.00557
2          0         -1           -0.01787   0.03121      113   -0.57   0.5680  -0.07971 ,  0.04396
3          0         -1           -0.02625   0.01729      113   -1.52   0.1318  -0.06051 ,  0.00801
4          0         -1           -0.10016   0.04406      113   -2.27   0.0249  -0.18745 , -0.01288
 
 
 ===========================================================================
 *****           Analysis 3: Random Readers and Fixed Cases            *****
 ===========================================================================
 (Results apply to the population of readers but only for the cases used in this study)


    a) Test for H0: Treatments have the same q
 
 Source        DF    Mean Square      F value  Pr > F 
 ----------  ------  ---------------  -------  -------
 Treatment        1       0.54676316     8.70   0.0420
 Error         4.00       0.06281753
 Error term: MS(TR)
 
 Conclusion: The treatment q
s are not equal,
             F(1,4.00) = 8.70, p = 0.0420.


    b) 95% confidence intervals for treatment differences
 
       Treatment         Estimate   StdErr      DF      t     Pr > t          95% CI      
----------   ----------  --------  --------  -------  ------  -------  -------------------
0          - 1           -0.04380   0.01485     4.00   -2.95   0.0420  -0.08502 , -0.00258
 
 H0: the two treatments are equal.
 
 

    c) Reader-by-case ANOVAs for each treatment (each analysis is based
       only on data for the specified treatment)
 

                        Mean Squares
 Source     df   0            1            
 ------    ---   ----------   ----------   
      R      4   0.35141971   0.14872460   
      C    113   0.33629383   0.16042320   
     RC    452   0.06043604   0.04403661   


 Estimates and 95% Confidence Intervals

  Treatment     Area      Std Error     DF     95% Confidence Interval 
  ----------  ----------  ----------  -------  -------------------------
  0           0.89703703  0.02482994        4  (0.82809807 , 0.96597599)
  1           0.94083734  0.01615303        4  (0.89598934 , 0.98568534)
 



                               REFERENCES

      Dorfman, D.D., Berbaum, K.S., & Metz, C.E. (1992). Receiver operating
 characteristic rating analysis: Generalization to the population of 
 readers and patients with the jackknife method. Investigative Radiology,
 27, 723-731.

      Dorfman, D.D., Berbaum, K.S., Lenth, R.V., Chen, Y.F., & Donaghy, B.A. (1998). 
 Monte Carlo validation of a multireader method for receiver operating characteristic 
 discrete rating data: Factorial experimental design. 
 Academic Radiology, 5, 591-602.

      Hillis, S.L., & Berbaum, K.S. (2004). Power estimation for the
 Dorfman-Berbaum-Metz method. Academic Radiology, 11, 1260-1273.

      Hillis, S.L., Obuchowski, N.A., Schartz, K.M., & Berbaum, K.S.
 (2005). A comparison of the Dorfman-Berbaum-Metz and Obuchowski-Rockette
 methods for receiver operating characteristic (ROC) data. 
 Statistics in Medicine, 24, 1579-1607  DOI:10.1002/sim.2024.

      Hillis, S.L. (2005). Monte Carlo validation of the Dorfman-Berbaum-Metz
 method using normalized pseudovalues and less data-based model simplification
 Academic Radiology, 12:1534-1541  DOI:10.1016/j.acra.2005.07.012.

      Hillis, S.L. (2007). A comparison of denominator degrees of freedom for
 multiple observer ROC analysis.  Statistics in Medicine, 26:596-619  DOI:10.1002/sim.2532.

      Hillis, S.L., Berbaum, K.S., & Metz, C.E. (2008). Recent developments in the
 Dorfman-Berbaum-Metz procedure for multireader ROC study analysis. Academic Radiology, in press.

================================================================================
