=========================
Overview and Introduction
=========================

Research of Semantic Web Services has gained a lot of momentum over the past few years, demonstrating its viability for dynamic service discovery, selection, composition, negotiation, mediation and invocation over the Web. However, design and development of Semantic Web Services tools and applications is still a new practice not only with the research community but also the industry. Software developers need to deal with new kinds of artifacts (e.g. semantic descriptions, ontologies, complex rules, formulas and facts), new kinds of operations (e.g. mapping, merging, transformation, querying and reasoning) and probably new kinds of user interactions and visualizations.

Best practices, methodologies and tools to help in dealing with these artifacts and operations in Semantic Web Services development are thus in high demand. Moreover, there are few scientific method of comparing and evaluating the strengths and weaknesses of functionalities claimed. Other than the SWS Challenge, there is no standard testbed to assess the robustness and applicability of proposed technologies to real world problem scenarios. Progress in scientific development and in industrial adoption in Service Oriented Architectures (SOAs) is thereby critically hindered.

The SWS Challenge initiative is specifically designed to develop a standard methodology and testbed for the evaluation of Semantic Web Services technologies. In addition to this this initiative, there is the associated W3C SWS Testbed Incubator which has issued a W3C Incubator Group Report about the evaluation methodology developed within the SWS Challenge.

This workshop, under the umbrella of the ISWC 2008 conference, will provide a common platform for researchers and developers in the field of Semantic Web Services to present the latest progress in the area and to discuss any gaps between the scientific research and required methodologies, tools and applications in practice.

First, it will focus on finding out different ways, methodologies, and technologies for evaluation of Semantic Web Services tools and applications, continuing the work of the SWS Challenge and the Incubator. Participants should be familiar with the previous work of these initiatives, including the Incubator Group Report above. Papers on this subject are highly welcome and need not be accompanied by a Challenge solution to be evaluated. Participants are invited to discuss the Challenge's current evaluation methodology. Insights gained during the workshops will provide additional input to the W3C SWS Testbed Incubator Group which aims at standardizing an evaluation methodology for SWS technology.

Second, the workshop will continue the SWS Challenge which is an ongoing and continuous experiment in evaluating the functionality of SWS technologies. The current set of evaluations are posted on the SWS-Challenge wiki (http://sws-challenge.org) and proposed solutions will follow the current evaluation methodology also available there. Participants are invited to propose new scenarios to be incorporated into the official SWS-Challenge testbed.

The activity most closely related to the SWS Challenge is the S3 Contest on Semantic Service Selection (http://www-ags.dfki.uni-sb.de/~klusch/s3/). However, the SWS Challenge and the S3 Contest are working in different parts of the reseach space. The organizers agree that they are related and both should be explored. In fact, the cross-referencing between both events on their web sites and inter-linkage of other activities (e.g., common use of test services) has been agreed between organizers.

==================
Topics of Interest
==================

* SWS Evaluation Methodolgy

- Methods to describe evaluation aspects and parameters
- Identification of resource consumption characteristics
- Models for describing and measuring in evaluation
- Techniques and formulas for evaluation
- Infrastructures and tools for evaluation
- Performance and scalability evaluations and benchmarks
- Integration of evaluation tools into frameworks
- Description of application of SWS technology in industry
- Real world use cases and evaluation metrics

* Semantic Web Services Challenge

- Descriptions of implemented solutions to any of the SWS-Challenge problem scenarios
- Proposals of new scenarios for evaluation
- Evaluation of surprise scenario variations

===============
Target Audience
===============

The Evaluation Methodology track of the workshop will address the researchers and practitioners working on evaluating and benchmarking Semantic Web Services technologies. It will also target potential users interested in learning about the feasibility of Semantic Web Services in market and industry.

The actual Challenge Evaluation track will address researchers and practitioners developing evaluation scenarios or solutions for the given problem set. The previous workshops have attracted not only people from the semantic web community in general, but also from the formal methods and software engineering community and from industrial research. The Challenge has thus succeeded to bridge communities working on different aspects in the field.

================
Workshop Program
================

The workshop requires two days. The 1st day will include the invited talk, presentations of accepted papers and panel discussion. The second day focuses on demonstrations of the scenario solutions and their evaluation. Typically, participants will be asked to develop solutions to "surprise" variations of scenarios overnight and present solutions on the 2nd day. Solutions of the variations will be verified and the participants will show that their solutions can be reversed so that the original problem can still be solved. There will be a group examination and evaluation of the code and/or methods required to do this, according to the most current methodology.

====================
Organizing Committee
====================

Chair:
* Charles Petrie, Stanford University, Palo Alto, USA

Steering Commitee:
* Dieter Fensel, Semantic Technology Institute (STI) International, Austria
* Michael Genesereth, DERI Stanford, USA

Program Committee Co-Chairs:
* Michal Zaremba, Semantic Technology Institute (STI) Innsbruck, Austria
* Emanuele Della Valle, CEFRIEL, Milan, Italy
* Ulrich Küster, University of Jena, Germany
* Tiziana Margaria, University of Postdam, Germany

Senior Advisory Board:
* Frank Leymann, University of Stuttgartt, Germany
* David Martin, SRI, USA
* Terry Payne, University of Southampton, UK
* Norman Sadeh , CMU, USA
* Amit Sheth, University of Georgia & Semagix, USA
* Rudi Studer, University of Karlsruhe, Germany
* Andreas Wombacher, University of Twente, The Netherlands

Special Advisor:
* Holger Lausen, Seekda OG, Austria

Technical Support:

Discovery Scenario Chair
* Ulrich Küster, Friedrich Schiller University Jena, Germany

Mediation Scenario Chair
* Federico Facca, Semantic Technology Institute (STI) Innsbruck, Austria

Infrastructure Chair
* Srdjan Komazec, Semantic Technology Institute (STI) Innsbruck, Austria

Logistics and Local Organization Chair
* Omair Shafiq, Semantic Technology Institute (STI) Innsbruck, Austria

=================
Program Committee
=================

* Sudhir Agarwal, University of Karlsruhe, Germany
* Anupriya Ankolekar, HP Palo Alto, USA
* Christoph Bussler, Merced Systems, Inc.
* Stefano Ceri, Politecnico di Milano
* Siobhán Clarke, Trinity College
* Oscar Corcho, Universidad Politecnica de Madrid, Spain
* Jos de Bruijn, University of Bolzano, Italy
* John Domingue, KMI, Open University, UK
* Peter Emmel, SAP, Germany
* Fausto Giunchiglia, University of Trento, Italy
* Matthias Klusch, DFKI, Germany
* Thorsten Liebig, Ulm University, Germany
* David Martin, SRI, USA
* Malgorzata Mochól, Free University Berlin, Germany
* Axel Polleres, DERI Galway, Ireland
* Kavitha Srinivas, IBM Research, USA
* York Sure, SAP Research, Germany
* Tomas Vitvar, STI Innsbruck, Austria
* Alexander Wahler, STI International, Austria
* Richard Waldinger, SRI, USA
* Baoshi Yan, Bosch (USA)

==========================
Submission and Proceedings
==========================

Submissions are to be formatted according to the guidelines of the ISWC2008 conference (Springer LNCS style). The page limit for full papers is 10 pages, for SWS-Challenge solution papers two additional pages are accepted.

Papers describing SWS-Challenge solutions should describe the technology employed for the solution and focus on how that technology was actually used to solve the challenge problems. The reader should be able to understand a particular solution in theory and practice by reading a solution paper. Additional submission of the code and documentation of the solution is strongly encouraged.

Note that ISWC requires workshop attendees to also register for the main conference. So try to register both for the conference and the workshop before the ISWC2008 early registration deadline.

Since the ISWC does not include workshop proceedings, we intend to publish the workshop proceedings online at CEUR-WS. If the number of high-quality submissions is high enough, we will pursue publishing the proceedings as a journal special issue or as printed proceedings.

* Guidelines for Paper Submission *
Papers are to be submitted online via the SWSC-ISWC2008 EasyChair submission system http://www.easychair.org/conferences/?conf=swsciswc2008. In case of any further questions about submission process, contact Omair Shafiq (omair.shafiq@sti2.at).

* Guidelines for Code Submission (only for SWS Challenge solution papers) * Interested participants should directly contact Srdjan Komazec (srdjan.komazec@sti2.at) to get an account for the SWS-Challenge FTP server (ftp://sws-challenge.org).

===============
Important Dates
===============

* Submissions due: 25 July 2008
* Acceptances: 5 September 2008
* Camera ready papers: 26 September 2008
* Camera ready proceedings: 03 October 2008
* Workshop dates: 26-27 October 2008 (to be confirmed)

Please do not hesitate to contact Charles Petrie for any questions you have!