TheoryEvalTechniques

Methods and techniques for evaluating e-consultation

This is the list of techniques suggested in the E-valuing E-consultation workshop at the 6th European Conference on E-Government in Marburg on 27th Apr. 2006. The instructions were:

List every evaluation methodology or technique you can think of that might be used to measure, assess or understand e-consultation.

1. Numbers
   1. Number of actual respondents
   2. Number of hits on the e-consultation site (Webstats)
   3. Range of types of participants
   4. representativity study of participants: are they a good sample of the affected population.
   5. Number of "e-consultation" words being cited in journal
2. Accounting
   1. Value for Money study
   2. Return on Investments (ROI)
   3. Return on relationship (ROR) measurement which includes indirect benefits ##Cost benefit analysis
   4. Impact analysis
   5. Did it work - did it bring up a solution?
3. Surveys
   1. Stakeholders satisfaction survey
   2. Satisfaction survey: a consultation worked if, after the complete cycle (once decisions were taken and participant received feedback) all involved parties -including the consulters- feel it was useful, think that the decission making will improve as a result of it and are willing to participate again on such process
   3. e-Poll on e-consultation website
4. Content analysis
   1. were all relevant points of view represented in the process?
   2. discourse analysis of questions, feedback
   3. Garrison's Theory of Critical Thinking (content analysis)
   4. Number of ideas used in preparation of documents discussing decision to be made
   5. social network analysis of communications among stakeholders in networked governance
5. Qualitative
   1. qualitative interviews with consulters and consultees before, during and after the process
   2. focus groups with consultees after the process
   3. Focus group with projective questions on the experience
   4. Structured interviews with decision makers
   5. real time blogs and/or forums
   6. get people to tell their story of the consultation on the city website
   7. interviews with range of LG officers on quality of participation
6. Long term
   1. longer term impact
   2. longitudinal study of return visits to consultations
   3. longditudinal study of citizens attitudes to a local government
   4. one year down the track return to consultees for reflection
7. Comparative
   1. construct ontologies of consultation types
   2. comparative studies with like consultations elsewhere
   3. deriving patterns from case studies

They need to be assigned to value categories.