Cem Kaner, J.D., Ph.D.

The Workshop on Teaching Software Testing is concerned with the practical aspects of teaching university-caliber software testing courses to academic or commercial students.

WTST 2013 is focused on high volume automated testing (HiVAT). Our goal is to bring together instructors who have experience teaching high-volume techniques or who have given serious consideration to how to teach these techniques. We also welcome participants focused on the teaching of complex cognitive concepts and the transfer of what was learned to industrial practice.

As at all WTST workshops, we reserve some seats for senior students who are strongly interested in teaching and for faculty who are starting their careers in this area or beginning a research program connected with teaching this type of material.

There is no fee to attend this meeting. You pay for your seat through the value of your participation. Participation in the workshop is by invitation based on a proposal. We expect to accept 15 participants with an absolute upper bound of 25.

BACKGROUND ON THE WORKSHOP TOPIC

High volume automated testing involves automated generation, execution and evaluation of the results of a large set of tests. This contrasts with more traditional “automated” testing that involves automated execution of a relatively small number of human-created tests.

Here are four examples of the types of problems that underlie the need for HiVAT:

1. Many types of code weakness (such as timing-related problems) yield intermittent failures and are hard to detect with traditional testing techniques
2. There are immense numbers of possible combination tests of several variables together. Some combinations are unique (a failure appears only on a particular combination)
3. Some failures occur primarily when a system under test is under load and so detection and characterization is essentially a statistical challenge.
4. Characterizing the reliability of software requires a statistically useful set of tests.

In the academic community, the most commonly discussed HiVAT family of techniques is called “fuzzing” – but fuzzing as we know it involves very simplistic evaluation of the test results—essentially run the software until it crashes or fails in some other very obvious way. Other HiVAT techniques rely on more powerful oracles and can therefore find other kinds of bugs.

WTST is about teaching testing, not creating new techniques. The challenge we are trying to address in this WTST is that many of these techniques are known but not widely applied. We believe this is because they are poorly taught. As far as we can tell, most testing courses don’t even mention these techniques. Of those that do (and go beyond fuzzing), our impression is that students come out baffled about how to actually DO that type of testing in their work.

At Florida Tech, we’re trying to address this by creating “reference implementations” for several techniques—open source demonstrations of them, with commentary on the design and implementation. We’re hoping that WTST will provide examples of other good approaches.

TO ATTEND AS A PRESENTER

Please send a proposal BY DECEMBER 1, 2012 to Cem Kaner <kaner@cs.fit.edu> that identifies who you are, what your background is, what you would like to present, how long the presentation will take, any special equipment needs, and what written materials you will provide. Along with traditional presentations, we will gladly consider proposed activities and interactive demonstrations.

We will begin reviewing proposals immediately. We encourage early submissions. It is unlikely but possible that we will have accepted a full set of presentation proposals by December 1.

Proposals should be between two and four pages long, in PDF format. We will post accepted proposals to http://www.wtst.org.

We review proposals in terms of their contribution to knowledge of HOW TO TEACH software testing. Proposals that present a purely theoretical advance in software testing, with weak ties to teaching and application, will not be accepted. Presentations that reiterate materials you have presented elsewhere might be welcome, but it is imperative that you identify the publication history of such work.

By submitting your proposal, you agree that, if we accept your proposal, you will submit a scholarly paper for discussion at the workshop by January 8, 2013. Workshop papers may be of any length and follow any standard scholarly style. We will post these at http://www.wtst.org as they are received, for workshop participants to review before the workshop.

TO ATTEND AS A NON-PRESENTING PARTICIPANT:

Please send a message by DECEMBER 1, 2012, to Cem Kaner <kaner@cs.fit.edu> that describes your background and interest in teaching software testing. What skills or knowledge do you bring to the meeting that would be of interest to the other participants?

The hosts of the meeting are:

Cem Kaner (https://kaner.com and http://www.testingeducation.org)
Rebecca Fiedler (http://bbst.info)
Michael Kelly (http://www.developertown.com/)

HOW THE MEETING WILL WORK

WTST is a workshop, not a typical conference. It is a peer conference in the tradition of The Los Altos Workshops on Software Testing (http://lawst.com). Our presentations serve to drive discussion. The target readers of workshop papers are the other participants, not archival readers. We are glad to start from already-published papers, if they are presented by the author and they would serve as a strong focus for valuable discussion.

In a typical presentation, the presenter speaks 10 to 90 minutes, followed by discussion. There is no fixed time for discussion. Past sessions’ discussions have run from 1 minute to 4 hours. During the discussion, a participant might ask the presenter simple or detailed questions, describe consistent or contrary experiences or data, present a different approach to the same problem, or (respectfully and collegially) argue with the presenter. In 20 hours of formal sessions, we expect to cover six to eight presentations. Some of our sessions will be activities, such as brainstorming sessions, collaborative searching for information, creating examples, evaluating ideas or work products. We also have lightning presentations, time-limited to 5 minutes (plus discussion). These are fun and they often stimulate extended discussions over lunch and at night.

Presenters must provide materials that they share with the workshop under a Creative Commons license, allowing reuse by other teachers. Such materials will be posted at http://www.wtst.org.

Our agenda will evolve during the workshop. If we start making significant progress on something, we are likely to stick with it even if that means cutting or time boxing some other activities or presentations.

LOCATION AND TRAVEL INFORMATION

We will hold the meetings at

Harris Center for Assured Information, Room 327

Florida Institute of Technology

150 W University Blvd

Melbourne, FL 32901

Airport

Melbourne International Airport is 3 miles from the hotel and the meeting site. It is served by Delta Airlines and US Airways. Alternatively, the Orlando International Airport offers more flights and more non-stops but is 65 miles from the meeting location.

Hotel

We recommend the Courtyard by Marriott – West Melbourne located at 2101 W. New Haven Avenue in Melbourne, FL.

Please call 1-800-321-2211 or 321-724-6400 to book your room by December 24, 2012. Be sure to ask for the special WTST rates of $93 per night. Tax is an additional 11%. All reservations must be guaranteed with a credit card by Tuesday, December 24, 2013. If rooms are not reserved, they will be released for general sale. Following that date reservations can only be made based upon availability.

For additional hotel information, please visit the hotel website at http://www.marriott.com/hotels/travel/mlbch-courtyard-melbourne-west/