Scientific progress relies on sound reviewing processes. The integrity of our review process is paramount, and it rests on our handling of ethical concerns and conflicts of interest. These issues have been compounded by the rapid growth in the computer architecture community, increases in reviewer load, and the introduction of virtual and hybrid conferences. To support our conference organizers, the SIGARCH and TCCA Executive Committees have developed and updated several written guidelines and best practices documents. These resources cover a range of strategic and tactical concerns for conference organizers, from committee formation and review assignments, to how to turn on logging and disable batch downloads in HotCRP. This blog post provides a guided tour of these resources.
There are several parties involved in the conference reviewing process, including the authors, the reviewers, and the program chairs. In the past, defining the responsibilities of various parties was a bit ad-hoc, varying significantly between conferences. The role of the program chair is especially difficult, as the person(s) assigned for the role may be doing this task for the first time, and may not be fully aware of the trade-off involved in various options for the several decisions associated with the reviewing process. Therefore, to ensure robustness, fairness, and consistency, SIGARCH and TCCA have jointly developed two documents. The first document, SIGARCH/TCCA’s Recommended Best Practices for Conference Reviewing Process. This document outlines the responsibilities of authors, reviewers, and PC chairs with respect to critical ethical issues, including conflicts of interest. A second document, SIGARCH/TCCA’s Recommended Best Practices for ISCA Program Chairs outlines various elements of the review process and decisions that specifically fall to the Program Chair. As with the earlier best practices, SIGARCH and TCCA encourage reviewers and PC chairs for any SIGARCH- or TCCA-sponsored conference, not just ISCA, to abide by the best practices outlined in these documents.
Conference organizers can also find the benefits and responsibilities of SIGARCH-affiliation in the Guidelines for SIGARCH Conferences. Sponsored conferences have the financial backing of the SIG and consequently have additional expectations relative to “in cooperation” conferences where the affiliation is looser. Similarly, IEEE CS offers financial sponsorship or technical sponsorship to conferences. In the former, an IEEE CS Technical Committee is financially involved and there are certain expectations on the conference. In the latter, there is no IEEE CS financial liability but still significant technical cooperation. For more details, see IEEE Policies, pages 76 and 77.
In conjunction with these policy guides, the SIGARCH and TCCA Executive Committees have released several public resources containing practical information for the community. These are living documents that will be updated regularly, to which the community is invited to contribute suggestions. The SIGARCH/TCCA’s Resource Packet for General Chairs aggregates advice and tactical information for general chairs. It also contains advice on critical tasks such as forming and vetting the conference organizing committee. The companion SIGARCH/TCCA’s Resource Packet for PC Chairs compiles tactical information related to that role, such as HotCRP configuration advice and submission deadline selection. Also useful for PC Chairs when forming the review committee, the Architecture PC Database (PCDB) is available online. This database aggregates publicly available program committee information for ISCA, MICRO, ASPLOS, HPCA, and IEEE Micro TopPicks dating back to 2014. We encourage our community to use this database to make our events more inclusive, distribute the review load, and broaden participation in the process.
About the Authors:
Martha Kim is an Associate Professor of Computer Science at Columbia University and a member of the SIGARCH EC.
Moinuddin Qureshi is a Professor of Computer Science at Georgia Tech, and an Executive Committee member of IEEE TCCA.