Reviewer Instructions 

The effectiveness of the workshop program depends on a knowledgeable, experienced, and ethical group of reviewers. It utilises a two-phase, double-blind review process where submissions are evaluated by a minimum of three reviewers - two of whom must be program committee members, and at least one must be a PhD student. All reviewers adhere to the ethics guidelines set forth by IEEE.

Good Review Practices 

Reviewers assess the trustworthiness, significance, and clarity of manuscripts. The results must be, primarily, accurate (trustworthy) and, secondarily, significant. A manuscript that is deemed trustworthy may be accepted even if it lacks significance. Conversely, a manuscript that is not trustworthy cannot be accepted, regardless of its apparent significance. Manuscripts that possess both trustworthiness and significance may receive priority. Furthermore, manuscripts must be sufficiently clear to facilitate the evaluation of their trustworthiness and significance. Reviewers are encouraged to undertake the following:

• Reflect upon and clearly state their limitations and biases.

• Distinguish between necessary changes and suggested modifications. It is preferable to delineate them distinctly.

• Identify sections of the paper that cannot be accurately evaluated or were not reviewed.

Reviewing Criteria 

The workshop expects significant, impactful, and potentially transformative contributions. Submissions will be evaluated based on their scientific quality and originality, potential impact, the extent to which the evidence substantiates the findings, the suitability of the methodology employed, and the rigour of the research process conducted.

Reviewers and authors must use the following checklist of essential quality attributes to evaluate manuscripts:

• Originality and level of innovativeness
  

• • The research method or results are direct repetitions, largely derivative, or very slight variations of prior work.

  • The research method or results are novel, demonstrating a clear attempt at original contribution and presenting a noticeable advancement over prior work.

• Description of the research method

• • The research method is either not described or poorly described; alternatively, the research method is understandable but lacks depth or detail. 

  • The research method — including what, where, when, and how data were collected and analysed — is sufficiently transparent to allow replication, with only minor omissions.

• Validity and reasoning

• • The research method is entirely unjustified, inappropriate, unclear, or incorrect, with little to no evidence supporting the claims; alternatively, the method may be somewhat valid but relies on limited or poorly matched evidence, flawed reasoning, minimal literature engagement, and/or overstated conclusions.

  • The research method is appropriate (though not necessarily optimal) for the stated purpose or problem and valid relative to literature/standards; conclusions are convincingly supported by evidence and logic, with transparent discussion of limitations and threats to validity.

• Applicability and impact

• • The research has no applicability or offers minimal mention of vague or speculative research, industrial, or societal impacts without support or context; alternatively, the research acknowledges potential practical implications or research impacts, but the reflection is underdeveloped.

  • The research convincingly demonstrates clear potential for real-world application and presents a well-developed reflection on possible research, industrial, or societal impacts.

• Clarity of exposition

• • The paper is unreadable, disorganised, or incoherent, with pervasive issues in language, structure, or logic; alternatively, the paper may be somewhat unclear, with a basic structure present but difficult to follow due to weak writing or organisation.

  • The paper clearly communicates its contributions, with polished writing, strong logical flow, rigorous explanations, consistent terminology, and an effective layout.

These essential quality attributes are necessary conditions for publishing the work. Without a compelling justification, a study that does not meet one or more essential quality attributes should not be accepted. Whenever a criterion is not met, ask yourself three questions:

• Does the paper justify the deviation from the essential quality attribute? AND

• Does deviating from the essential quality attribute in this way make sense in the context of this study? AND

• Would this problem be easy to fix in the camera-ready copy?

  > Justified deviations and easy-to-fix mistakes should not preclude acceptance.

  
  

Therefore, reviewers do not provide an overall recommendation; whether to accept or reject a paper is a consequence of the recommended decisions based on the logic below. Instead, the Technical Program Chair compiles the reviewers’ evaluations and informs the authors whether the paper is accepted, requires revision and resubmission, or is rejected.

If you decide on No for any essential quality attribute, you must provide a justification for your evaluation or specify what would be required for revision in the free-form comments section.

Authors and reviewers must use this checklist of essential quality criteria because it:

• Makes peer review less biased,

• Reduces review loads by reducing the need for major revisions and overall reviewing work,

• Increases acceptance rate because expectations are known in advance,

• Reduces the reviewers' extraneous cognitive load associated with generating ad hoc criteria for each review,

• Actively prevents reviewers from imposing their own, non-standard criteria, and

• Mitigates the frustration caused by differences between the author's and reviewers' expectations,

Review Feedback

• If the review decision is "accepted", as a reviewer, you should provide authors with a review summary.

• If the review decision is "revise and resubmit", as a reviewer, you should provide authors with a to-do list.

• If the review decision is "rejected", as a reviewer, you should provide authors with a summary of the reasons for rejection.

Form and Shape of Review 

• Constructive criticism: The review should be constructive and give indicators for improvement.

• Factual statements: Statements made by the reviewer should be factual and backed up / justified.

• Clarity: The statements and recommendations should be clear and detailed.

• Transparency: The decision/recommendation to the chair should become evident from the statements and arguments in the review.

• Reasonable suggestions: Suggestions made should be reasonable and feasible.

• Comprehensive: Review should be in-depth rather than being a short, non-helpful statement.

• Soundness: Review should be sound and coherent.

• Professional tone: Review should be written in a professional, non-emotional tone.

Conduct and Attitude 

• Accuracy: The reviewer took care to read the paper and provide a review that is factually correct relative to the contents of the paper.

• Fairness: The reviewer should be fair and carefully consider both negative and positive points.

• Expertise: The reviewer should have clear expertise on the topic.

• Openness: The reviewer should be objective and open to new ideas that may not align with their own belief system.

• Focus on soundness: The reviewer should focus on methodological soundness rather than on actual results.

• Honesty: The reviewer should be honest about views and expectations.

Reviewing Antipatterns 

Invalid Criticisms 

• Indicating that a study:
• Lacks detail without listing the specific missing details;
• Is absent of crucial references without identifying which references are lacking;
• Is of insufficient quality without elucidating particular issues;
• Is unoriginal without citing published studies that offer substantially identical contributions.

• Critiquing a study based on limitations inherent to that type of research or the employed methodology (for instance, disputing a case study due to its limited generalisability).
• Rejecting a study due to the reviewer's preference for an alternative methodology or design.
• Rejecting a study because it presents negative results.
  

Reviewer Confidentiality Obligation 

IEEE receives papers in confidence and protects the confidentiality of their contents until they are published. As a reviewer, you are obligated to maintain the confidentiality of both the paper you are reviewing and also of your review.

IEEE Policy on Reviewers Using AI 

Information or content contained in or about a manuscript under review shall not be processed through a public platform (directly or indirectly) for AI generation of text for a review. Doing so is considered a breach of confidentiality because AI systems generally learn from any input.