This file includes the detail for each submission in study CSF_HSa, each row represent one property.
Explanations for each column:
- WorkerID: participant's worker id issued by Amazon obfuscated by us using our own hashmap function.
- Library: The library we explored in Java.
- Method1: The first method in the LTL property.
- Property: LTL properties include "AF","NF","AP","AIP","AIF","NIF".
- Method2: The second method in the LTL property.
- Answer: Participant's answer for certain property.
- Confidence: Participant's confidence for the answer.
- Explanation: Participant's explanation for the answer.
- SpecForge: The answer from SpecForge for a property.
- GroundTruth: The real answer for a property.
Experimental raw data for CSF_HSa