How to Analyze Multi-Observation Experiments That are Observed Periodically (Repeated Measure)

When conducting field research, repeated observations are often needed to understand the dynamics of change over time. SmartstatXL recognizes this need and offers a comprehensive solution for multi-observation data analysis. This allows researchers to collect data from the same experimental unit at different time points or conditions, providing a more in-depth view of the variable response to treatments.

Types of multiple observations include:

• Sub-Sampling: This relates to measurements taken on multiple samples from the same experimental unit. For example, in plant growth research, the height of 10 plants may be measured in a single experimental unit to provide a more accurate representation of average growth.
  
• Repeated Measure: Here, the same variable is measured at different time intervals. This is extremely useful in research that aims to track changes over time, such as the progression of plant growth stages or responses to specific stimuli.
  

With SmartstatXL, researchers can not only efficiently process their data but also interpret it correctly. If the analysis shows a significant difference between treatments, various Post Hoc Tests are available to understand those differences in greater detail. Available options include: Tukey, Duncan, LSD, Bonferroni, Sidak, Scheffe, REGWQ, Scott-Knott, and Dunnet. With these tools, researchers can ensure that their interpretation is supported by precise and thorough statistical analysis.

Case Example

An experiment was carried out to study the differences in yield of four alfalfa cultivars. Five replications of these four varieties were organized according to a Completely Randomized Design (CRD), and four cuttings were made of each replication over time. The data represents the repeated measurements of yield (tons/acre) of the four cultivars:

An experiment was conducted to study the yield differences among four alfalfa cultivars. Five replications of these four varieties were arranged in a Completely Randomized Design (CRD), and four cuttings were performed for each replication over time. The data represent the repeated measurements of yield (tons/acre) from the four cultivars:

• Variety: Indicates the alfalfa variety being tested (e.g., variety 1, variety 2, etc.).
• Replication: Indicates the replication for each variety (e.g., replication 1 for variety 1, replication 2 for variety 1, etc.).
• Yield Cutting 1, Yield Cutting 2, Yield Cutting 3, Yield Cutting 4: Indicates the yield at the first, second, third, and fourth cuttings.

Cited from:
Gomez, Kwanchai A. and Gomez, Arturo A. 1995. Statistical Procedures for Agricultural Research. [translator] Endang Sjamsuddin and Justika S. Baharsjah. Second Edition. Jakarta: UI-Press, 1995. ISBN: 979-456-139-8. p. 350.

Steps for Analysis of Variance (Anova) and Post Hoc Tests:

1. Ensure that the worksheet (Sheet) you want to analyze is active.
2. Place the cursor on the Dataset. (For information on creating a Dataset, please refer to the 'Data Preparation' guide).
3. If the active cell is not on the dataset, SmartstatXL will automatically detect and select the appropriate dataset.
4. Activate the SmartstatXL Tab
5. Click on the Menu CRD/RBD > Repeated Measure (In this case example, SmartstatXL uses RBD).
   
6. SmartstatXL will display a dialog box to confirm whether the Dataset is correct or not (usually the cell address for the Dataset is automatically selected correctly).
   
7. After confirming the Dataset is correct, press the Next Button
8. Next, a dialog box titled Anova – Single-Factor RBD will appear (observed periodically).
   
   The One-Factor Experimental Model is analyzed by adding an additional Time Factor and analyzed simultaneously (sometimes called Mixed Design or Split Plot in Time).
   In the Split Plot in Time model, for both CRD and RBD, the Replication Factor must be included in the model!
   Compare with the One-Factor CRD Experimental Model that is analyzed partially on each cutting (Cutting), as in the following image:
   
   In the above One-Factor RBD Experimental Model, the variables included in the Model (Factors) are only Treatment and Group.
9. There are 3 stages. The first stage, select the Factor and at least one Response to be analyzed (As shown in the image above)!
10. When you select a Factor, SmartstatXL will provide additional information on the number of levels and the names of those levels.
11. Details of the Anova STAGE 1 dialog box can be seen in the following image:
    
12. After confirming the Dataset is correct, press the Next Button to proceed to Anova Dialog Stage-2
13. The dialog box for the second stage will appear.
    
14. Adjust the settings based on your research method. In this example, the Post Hoc test used is Tukey's test.
15. To set additional output and default values for the next output, press the "Advanced Options…" button
16. Here is the appearance of the Advanced Options Dialog Box:
    
17. Once the settings are complete, close the "Advanced Options" dialog box
18. Next, in the Anova Stage 2 Dialog Box, click the Next button.
19. In the Anova Stage 3 Dialog Box, you will be asked to specify the average table, ID for each Factor, and rounding of the average values. The details can be seen in the following image:
    
20. As the final step, click "OK"

Analysis Results

Analysis Information

Experimental Design

This analysis employs a Randomized Complete Block Design (RCBD) with a single factor repeated periodically. In this context, the single factor refers to the 'Varieties' of alfalfa, with five replications for each variety. Additionally, four cuttings are performed for each replication over time.

Post Hoc Test

The Post Hoc test used is Tukey (HSD), commonly employed for comparing group means in Analysis of Variance (ANOVA).

Responses and Factors

1. Response: The response variable in this analysis is 'Yield,' or harvest output in tons per acre.
2. Factors:
   • Replication: With 5 levels.
   • Varieties: With 4 levels.
   • Cutting: With 4 levels.

Assumption Violations and Solutions

In Analysis of Variance, certain assumptions must be met, such as homogeneity of variances and normal distribution. In this case, these assumptions seem to have been violated ('--Assumption Violations--').

As a solution, logarithmic transformation was applied to the 'Yield' data. This transformation is often used to stabilize the variance and make the data distribution more normal. Additionally, outlier data were replaced with values from missing data calculations.

Therefore, SmartstatXL has carefully prepared features to address assumption violations, either through data transformation or appropriate statistical method selection. Next, we will discuss how the analysis of variance can provide insights into differences in alfalfa yield between varieties, between replications, and over time (cutting).

Analysis of Variance

Interpretation and Discussion: Analysis of Variance Results for Original Data

• Varieties (V): Varieties show a significant difference at a 1% significance level (F-Calculated > F-0.01 and P-value < 0.01). This means that there are significant yield differences between the four alfalfa varieties tested.
• Cutting (C): Cutting time also shows a significant difference at a 1% significance level (F-Calculated > F-0.01 and P-value < 0.01). This indicates that the yield differs significantly at the four different cutting times.
• V x C (Variety and Cutting Interaction): There is no significant interaction between variety and cutting time (F-Calculated < F-0.05 and P-value > 0.05). This means that the effect of variety on yield is not dependent on the cutting time, and vice versa.

Coefficient of Variation

• CV(a): 11.95%
• CV(b): 10.33%

The coefficient of variation (CV) indicates how much the data varies. In this context, CV(a) and CV(b) are relatively low, indicating that the data is quite consistent.

Conclusion

• There is a significant difference in yield between different varieties.
• There is a significant difference in yield at different cutting times.
• There is no significant interaction between variety and cutting time on yield.

Therefore, both variety and cutting time are factors that need to be seriously considered in improving alfalfa yield. However, both factors operate independently and do not influence each other.

Post Hoc Tests

Single Effect of Varieties (V)

Critical Values and Standard Error

• Standard Error: 0.0800
• Tukey (HSD) 0.05: 0.3237

The standard error is a measure of how much the sample mean differs from the population mean. The Tukey (HSD) value at the 0.05 significance level is used for comparing means between two varieties.

Interpretation

• Variety 1 has an average yield that is not significantly different from varieties 3 and 4 according to the Tukey Post Hoc Test at the 0.05 significance level (labeled 'ab').
• Variety 2 has a higher average yield that is significantly different compared to varieties 3 and 4 (labeled 'b').
• Varieties 3 and 4 are not significantly different from each other in terms of yield (labeled 'a').

Conclusion

• Variety 2 shows higher yield potential compared to varieties 3 and 4.
• Variety 1 shows more flexible potential as its yield is not significantly different from varieties 3 and 4.

Therefore, in the context of increasing harvest yield, variety 2 could be a promising option. Meanwhile, variety 1 should also be considered as it shows similar yield potential to other varieties, depending on other factors such as production cost or specific needs.

Single Effect of Cutting (C)

Critical Values and Standard Error

• Standard Error: 0.0692
• Tukey (HSD) 0.05: 0.2607

The standard error is a measure of how much the sample mean differs from the population mean. The Tukey (HSD) value at the 0.05 significance level is used for comparing means between two cutting times.

Interpretation

• Cutting 1: Has a higher average yield compared to Cutting 4 but is lower compared to Cutting 2 and Cutting 3 (labeled 'b').
• Cutting 2: Has the highest average yield compared to all other cuttings (labeled 'd').
• Cutting 3: Has a higher average yield compared to Cutting 1 and Cutting 4 but is lower compared to Cutting 2 (labeled 'c').
• Cutting 4: Has the lowest average yield compared to all other cuttings (labeled 'a').

Conclusion

• Cutting 2 shows the most optimal yield compared to other cutting times.
• Cutting 4 shows the lowest yield and may need to be avoided or improved through other interventions.

This means that the timing of the cutting is a crucial factor in determining the yield of alfalfa. Therefore, choosing the right cutting time can be an effective way to increase productivity.

Effect of the Interaction Between Variety and Cutting

Assumption Checks

Formal Approach (Statistical Tests)

Levene's Test for Homogeneity of Variances

• Degree of Freedom 1 (DF₁): 15
• Degree of Freedom 2 (DF₂): 64
• F-Value: 6.07
• P-Value: 0.000

Levene's test is used to test the homogeneity of variances between groups. In this case, a significant P-value (less than 0.05) indicates that the variances between groups are not homogeneous. This is a violation of one of the main assumptions of ANOVA.

Normality Tests

• Various statistical tests are used to test the normality of the distribution of residuals:
• Shapiro-Wilk's: Statistic 0.961, P-Value 0.015
• Anderson-Darling: Statistic 0.780, P-Value 0.043
• D'Agostino-Pearson: Statistic 16.635, P-Value 0.000
• Liliefors: Statistic 0.108, P-Value less than 0.05
• Kolmogorov-Smirnov: Statistic 0.108, P-Value greater than 0.20

Except for the Kolmogorov-Smirnov test, all other tests show a significant P-value (less than 0.05), indicating that the distribution of residuals is not normal. This is also a violation of the assumptions of ANOVA.

Conclusion

Both main assumptions of ANOVA—homogeneity of variances and normal distribution—have been violated in this analysis. Therefore, the results from ANOVA must be interpreted cautiously. In situations like this, remedial measures are usually required, such as data transformation or the use of non-parametric statistical methods.

However, these violations do not always undermine the validity of the findings, especially if the sample size is large enough. Nevertheless, these assumption violations require extra caution in interpreting the results and considering alternative analysis methods.

Visual Approach (Plot Graphs)

Interpretation and Discussion

1. Normal P-Plot of Residual Data
   • The Normal P-Plot is used to assess whether the data is normally distributed. In an ideal graph, the data points would follow the diagonal line. From the graph, it's evident that the data points tend to deviate from the diagonal line, indicating a potential violation of the normality assumption.
2. Residual Data Histogram
   • The histogram is used to depict the frequency distribution of data. In this case, the shape of the histogram indicates the presence of skewness and possibly kurtosis, both of which are additional indicators of normality violation.
3. Residual vs. Predicted Plot
   • This graph is used to evaluate homoscedasticity, which is the homogeneity of residual variances. In an ideal graph, points would be randomly and evenly distributed around the horizontal zero line. However, from this graph, a pattern is evident, indicating the presence of heteroscedasticity or a violation of the homogeneity of variances.
4. Standard Deviation vs. Mean
   • This graph shows the relationship between the standard deviation and the mean for each group. Ideally, points would be randomly distributed without any particular pattern. However, a pattern is evident in this graph, which could indicate a violation of the homogeneity of variances assumption.

Conclusion

The above graphs indicate violations of several key ANOVA assumptions, including normality and homogeneity of variances. Therefore, corrective measures need to be taken before proceeding with the interpretation of ANOVA results or considering the use of alternative, more robust analysis methods against assumption violations.

Box-Cox and Residual Analysis

Box-Cox Transformation

• Lambda: 0.042
• Transformation: Log Transformation: Log(Y) or LN(Y)

The Box-Cox transformation is used to correct violations of normality and homogeneity of variances in ANOVA. A Lambda value of 0.042 suggests that a logarithmic transformation (Log(Y) or LN(Y)) is the most appropriate method to use.

Residual Values and Outlier Examination

The columns in this table include:

• Replication, Variety, Cutting: Indicate the factors used in the experiment.
• Yield: The original harvest results.
• Predicted: The harvest results predicted by the model.
• Residual: The difference between the original and predicted harvest results.
• Leverage: The measure of how much a data point influences the parameter estimation of the model.
• Studentized Residual and Studentized Deleted Residual: The normalized residuals.
• Cook's Distance: The measure of how much a data point influences the entire model.
• DFITS: Another diagnostic similar to Cook's Distance.
• Diagnostic: Indicates whether the data point is an outlier or not.
• Box-Cox Data: Data after the Box-Cox transformation.

Interpretation

• Data Points with 'Outlier' Diagnostic: There are several data points identified as outliers, such as in replication 1 for variety 4 and cutting 2 with a Cook's Distance value of 0.1495, and replication 2 for variety 2 and cutting 2 with a Cook's Distance value of 0.1126. These outliers can affect the analysis results and may need to be further analyzed or removed.

Conclusion

The Box-Cox transformation was successfully used to correct violations of the assumptions of normality and homogeneity of variances. Additionally, the examination of outlier data revealed several points that might affect the analysis results. Corrective measures may need to be taken, such as removing or adjusting these outliers, before proceeding with further analysis.

Data Transformation

Interpretation and Discussion: New Data Meeting ANOVA Assumptions

Logarithmic transformation (Log(Y) or LN(Y)) and replacement of outlier data with values from missing data calculations have been performed to ensure that the data meets the ANOVA assumptions. This is important because violations of assumptions can affect the accuracy and reliability of the analysis results.

New Data Table:

The columns in the table include:

• Replication, Variety, Cutting: Experimental factors.
• Yield: The original harvest results.
• Yield*): Data for the harvest results after logarithmic transformation and/or outlier data replacement.
• "Replace Outlier Data: Yield": Indicates whether a data point is an outlier and has been replaced.

Interpretation

• Replaced Data: Some outlier data points have been replaced, such as in replication 1 for variety 4 and cutting 2, and replication 4 for variety 2 and cutting 2. These data points have been replaced using missing data calculations to ensure analysis accuracy.
• Logarithmic Transformation Data: The entire "Yield*)" column shows harvest results after logarithmic transformation, aiding in meeting the assumptions of normality and homogeneity of variance.

Conclusion

By performing data transformation and replacing outlier data, the dataset is now more suitable for ANOVA analysis. These steps help ensure that the analysis results will be more accurate and reliable. Subsequently, Analysis of Variance (ANOVA) can be performed on this dataset to understand the effects of experimental factors on harvest results.

ANOVA Assumption Checks for New Data

1. Levene's Test for Homogeneity of Variance

Levene's test is used to check the homogeneity of variance between groups. In this case, the P-value is greater than 0.05, meaning there is not enough evidence to reject the null hypothesis that the variance between groups is homogeneous. This is a good indicator that the data meets one of the key ANOVA assumptions.

2. Normality Test

Various statistical tests are used to check the normality of the residual distribution. All tests show a P-value greater than 0.05, indicating that there is not enough evidence to reject the null hypothesis that the residuals are normally distributed. This suggests that the data meets the normality assumption, another key assumption of ANOVA.

Conclusion

After performing logarithmic transformation and replacing outlier data, it appears that the data now meets the ANOVA assumptions, including homogeneity of variance and normality of distribution. This paves the way to proceed with ANOVA analysis, with the expectation that the results will be more accurate and reliable.

Analysis of Variance Results and Post hoc Tests

Other post hoc tests are not included here.