We validated the results of the comparative analysis through an expert questionnaire survey complemented by targeted follow-up interactions. Experts evaluated our findings in terms of completeness, correctness, and alignment with established practice.
The evaluation focuses on validating the results reported for RQ1-RQ3. The survey includes Likert-type questions and structured qualitative feedback to support the interpretation of disagreements and to refine the analysis where needed.
Invitations sent
Complete responses
Response rate
Responses were collected over four weeks (January 2026). The respondent group included a mix of academic and industry profiles (e.g., PhD students, postdocs, researchers, professors, and practitioners), with an average experience of approximately 8 years in robotic software engineering. Follow-up interactions were conducted in February 2026.
Note: We collected informed consent from all respondents prior to participation, ensuring that they understood the purpose of the study and their rights regarding data usage.
Experts evaluated the mapping between formalism primitives and the control structures / mission concepts. Feedback was used to confirm or refine the interpretation of constructs and the consistency of the mapping.
Most respondents agreed with the expressed support levels and the identified strengths/weaknesses. Disagreements mainly concerned concerns that can be addressed through implementation-level extensions (e.g., time-dependent behavior, state saving & resuming, explicit waiting), and were discussed and integrated into the final observations.
Experts validated the tool taxonomy (scope and capabilities), reported their familiarity with the tools, and suggested missing tools or overlooked aspects. Feedback informed refinements of the tool tables and discussion.
Questionnaire: Link to questionnaire
Responses: Open the spreadsheet in a new tab
Follow-up transcripts: par:5 | par:10 | par:17 | par:20
Note: transcripts are anonymized and shared for replication purposes.
Anonymized responses spreadsheet (read-only).