CrowdSpec: Supporting materials

Paper title: Mining Specifications from Documentation Using a Crowd

User studies

We ran several studies to evaluate CrowdSpec on the three Java APIs. These studies are described in the paper. On this page we make available the complete ground truth that we manually derived as well as the raw results from the studies we ran.

Ground truth

• Ground truth temporal specifications for HasSet API (1,014 properties in total)
• Ground truth temporal specifications for StringTokenizer API (384 properties in total)
• Ground truth temporal specifications for StackAr API (600 properties in total)

SpecForge true properties

• SpecForge true properties for HasSet API (26 properties in total)
• SpecForge true properties for StringTokenizer API (27 properties in total)
• SpecForge true properties for StackAr API (34 properties in total)

Qualification Test

• Qualification test

HIT Design

• HIT design

Experiments to answer research questions

• HashSet_A (CSF_HSa)
• Experimental artifacts

• HashSet_B (CSF_HSb)
• Experimental artifacts

• StringTokenizer (CSF_ST)
• Experimental artifacts

• StackAr (CSF_SA)
• Experimental artifacts