Notes on Organizing ICSE Workshops

July 7, 2000

This document captures information about our experiences organizing the workshops program for ICSE 2000. In addition to "how-to" information, we have tried to identify what went well and what we would do differently if we were to do it again! Since this is mostly a retrospective document, we are sure we haven't captured everything!

Basic Timeline

We signed on as workshops co-chairs approximately two years before the date of the conference. Here is a basic list of tasks we performed and when the tasks were performed.

May 1998	Signed on as co-chairs
June 1998	Provided initial list of activities and dates. Here is a list of activities identified: Prepare text for call for submissions for publicity chair and web site Decide on workshop proposal submission date Decide on common date for workshop submissions (this is to individual accepted workshops) Decide on date for acceptance/rejection notification for workshop proposals Prepare publicity for accepted workshops Prepare information on room/a/v requests for local organizers Workshop text for advance/final program
August 1998	Activities list was refined to include dates for drafting and having acceptance/notification letters approved.
September 1998	Drafting of early CFP.
January 1999	Drafting of CFP.
May 1999	ICSE '99. Planning meetings. We also started talking to ICSE participants about submitting a workshop proposal.
August 1999	We formed our "workshop propsal review committee" which consisted of ourselves and two outside reviewers.
September 1999	Reminder e-mail to those we had approached about submitting proposals but hadn't yet submitted about the impending date of submission.
September 1999	Contacting submitters for missing bits of proposals.
September 1999	Kept general chair and tutorials chair updated about list of submissions.
October 1999	Begin workshop selection process. Start finalizing with EC the target number of workshops (given targets for tutorials, room availability, etc.)
November 1999	Determined acceptances/rejections. Sent out notices. Asked accepted workshop organizers to confirm with some information about constraints, their AV requirements, etc. Asked to provide write-up about accepted workshops to SEN.
December 1999	Liasion to get workshop information on ICSE web site.
January 2000	Decided to postpone the workshop common submission deadline (mainly because of delay to put advanced program on the web). Notified of the workshop fees. Discussed whether to get industrial sponsorship for workshops to guarantee strong participation (not done, and aposteriori it was not needed). Discussion of how to manage workshop registrations (i.e., was there a need to cross-check registration for closed workshops, monitor the number of registrations wrt the expected number, etc.)
February 2000	OC Meeting in Wien: decided a contribution of 2 pages for each workshop to go into ICSE Proceedings. Decided the common time table for all workshops (i.e., start time, breaks, lunch). Decided one same cover for all workshop proceedings, with a rectangular hole where to put the title.
March 2000	Collected the 2-page contributions for ICSE Proc. from workshop organizers and an overview by us, and sent to proceedings editor. Distributed the format of the workshop proceedings roc. cover to organizers, with some following discussion on e-mail. One workshop was unable to fit their title in the cover. The compromise was for them to produce their own proceedings with a negotiated cost reimbursed by ICSE. Negotiated with organizers of one of the workshops who wanted to cancel it, finally decided to only reduce its duration from two to one day only. The question arose of how to guarantee that invited participants get a place for registering, in view of the upper-limit. Based on a request by Will Tracz, sent e-mail to all oganizers inviting them to submit a summary of their workshop to SEN.
April 2000	Workshop organizers asked to provide estimate for the number of pages needed for their respective notes.
May 2000	Final Program: check contents. Workshop organizers send camera-ready proceedings, either in electronic or paper format, to local organizers.
June 2000	A successful conference!
July 2000	Follow-up e-mail to organizers on submitting summary to SEN.

What Went Well

Workshop Proposals. We brainstormed a number of potential topics for workshops and approached several people to invite them to submit a proposal (the proposal was still subject to review). We also contacted organizers of successful workshops from the past ICSE to invite them to submit. This seemed to work well. We received 36 submissions and selected 16. One caveat is to make sure that anyone you invite to submit knows that their proposal will be reviewed (and might not be accepted).

Workshop Selection. We recruited two outstanding volunteers to help us review the workshop submissions. We made up a small form to ensure the reviews were in similar formats. The "committee" approach to selection and the "forms for review" were both invaluable! We also tracked submissions via a web page to provide visibility. Both of the co-chairs ended up receiving workshop submissions. Care must be taken not to lose any! (Perhaps it would be better if all submission went to one chair.)

Common workshop submission deadline. All in all, having a common submission deadline for all of the workshops worked well. It made publicity about the workshops easier. But, there is no way to force organizers of individual workshops to respect this date. We didn't have any trouble with that but one should just be aware! When picking this date, be careful that there is enough time for people submitting to the workshops to be notified of acceptance/rejection and still respond to the early registration date. (There was one problem. Check below.)

What Could Have Gone Better

Common workshop submission deadline. One problem we did run into with the common submission deadline was that it took longer than expected (more on the order of months than days) to get the accepted workshop information onto the web page. As a result, we had to slide the submission date. This is a problem because there is a cascading effect of messages going around to SEWorld and other places announcing new dates. It also meant that the acceptance/rejection date was much closer than desired to the early registration date. (For instance, getting a Euro bank draft to Ireland to register was not easy to do for Gail's students!)

Workshop Selection. Although we asked the organizer(s) of the previous ICSE workshops about the selection process, etc., we neglected to ask if there were any issues which arose regarding particular workshops. Because of the limitations on the number of workshops and the number of good proposals, in some cases, we asked workshops to merge (prior to selection). In one case, this worked out fine. In a second case, it turned out the workshops had been asked to merge the year before and it had been impossible. Managing this was not a lot of fun. (And we didn't have a lot of time lee-way in which to try this merging.) We ended up accepting both workshops for less time than they had requested.

Formats. The organizers of the workshops were requested to submit several different things. The original proposal (make sure to request what format you want---we got a range from Ascii to Word to PDF), summaries for the proceedings (using ICSE conference proceedings format), and the proceedings for the workshops (using yet another format). There was confusion between the ICSE conference proceedings format (for the summaries in the proceedings) and the workshop proceedings format. One particular problem was that the conference proceedings format used letter format while the workshop proceedings had to be in A4. Another problem that arose (late) was that the workshop proceedings needed to be numbered, but the organizers had collected position papers to form the proceedings in different formats: numbering these is hard!

Workshop Proceedings. For ICSE 2000, the conference itself reproduced proceedings provided by each workshop organizer. We didn't specify a length on these proceedings. Some ended up being larger than expected (long position papers with lots of participants). A length should have been communicated to the workshop organizers before they solicited position papers.

Scheduling Workshops. Remember to ask in the call for proposals for any constraints the organizers have on giving the workshop on specific days. The workshops for ICSE 2000 were split across four days. Some organizers were grumpy that their workshop was scheduled such that there was a day break before the conference started, but this was unavoidable.

Coordinating the workshop program with other ICSE tracks It is desirable to coordinate the workshop program with other events, notably tutorials, for avoiding conflicts and overlaps. This year this was not thought out in advance and the schedule was not set so as to facilitate interaction:: the workshop acceptance deadline was 12/11, while the deadline for tutorial submission was 11/11. We had to decide the program in front of just hyphotetical scenarios for tutorials. Remember to schedule workshops in conjunction with tutorials to try and avoid overlap!

Call for Proposals. Remember to ask in the call for proposals for any constraints the organizers have on giving the workshop on specific days. Also, ask for the abstract up-front to save time and effort when sending out acceptances.

Workshop deadlines wrt ICSE deadline. The date for informing submitters whether or not their workshop proposal was accepted was right at the time ICSE papers were due! Getting your own paper finished and dealing with the workshop organizers at the same time was not easy!

Workshop Identification. Once the workshops are selected, they are identified with a number. After this, all the communications about workshops must refer to these numbers. But organizers rarely did this: they just sent out e-mail asking something and saying "regards, John, or Mike...and they assume that everyone readily knows what they are talking of. Therefore several times we were resent messages from the local organizers asking "which Workshop is John's, or Mike's". Make clear with organizers that they will be identified by their number (this did not work for us, though).

Memory of previous issues. We asked the chair a previous ICSE Workshops program to give us some input and feedback about this task. The insights provided were very valuable, but these were given informally. We hope that saving some notes such as these for future organizers will be useful.

Questions/Issues That Arose

Workshop Attendence Fees. We had some complaints from organizers that they didn't feel that neither they nor special invitees to a workshop should have to pay the registration fee. It turns out that invited workshop speakers showed up at the conference without having registered. This issue must be decided early and communicated to the organizers at the very beginning. (Although we did communicate this early, it didn't seem to help :( Perhaps there is some better way.).

Number of Workshops. The number of workshops that can be held is (obviously) constrained by the number of rooms available. It is also constrained by organizational capacities. All in all, the registration at the workshops was good so having quite a few workshops worked out well.

Workshop Summaries in Proceedings. We did put two-page summaries of workshops in the proceedings. One problem is that not all workshop organizers will respond to the request. Another problem is formats (see above). Also, the contribution is collected electronically, but ACM wants a signed copyright form in paper.

Workshop Proceedings. For ICSE 2000, the conference itself reproduced proceedings provided by each workshop organizer. There was some problems with formats (see above) and lengths (see above). Some workshops will want to liasion with a publisher to publish their proceedings. We were not involved in such cases.

Tradition. Because we wanted to keep the workshop program "fresh", we tried to select a combination of workshops which had a strong tradition at ICSE and "new" workshops. Its hard to turn down workshops which have a long tradition (but we did).

Closed versus Open Workshops. Some workshops were "closed" (i.e., invited participants only) and some were "open" (some invited participants but all could register). It was determined in cooperation with the EC that the registration of closed workshops would be "checked" by the registration company. To permit this, workshop organizers were to provide a list of possible registrants to the registration company. The problem was that registrations started to arrive before the lists from the organizers and the registration company had a policy to reply to a registration in 48 hours. As a result, some people were allowed to register for otherwise closed workshops. This needs to be managed better in the future. Also, even though we had asked the organizers to check lists of "closed" vs. "open" settings for workshops we sent around, there were still some workshops for which it was desired to change from "open" to "closed". Watch out for this!

Comments on Workshop Proposals. We didn't provide review comments on the workshop proposal submissions back to the organizers. In only one or two cases were we asked for such comments. We just summarized the comments in these cases. This approach worked out well. Workshops are difficult to review so its hard to send something back to the proposers.

Workshop Lengths. We had many proposals for workshops that were to be 2 days. In some cases, we asked the organizers to shorten the length. One problem with long workshops is that it reduces the total number you can accept.

Confidentiality issues. It may be often the case that some of the workshop proposers are within the ICSE OC, this year it was so. We had e-mail discussions on the OC mail list to finalise the program before the notification was sent to the proponents, and therefore those who had submitted were involved in the discussion to finalise the program. It should be decided in advance whether or not this is desirable.