[Balug-admin] evaluation data, etc. (2005-10-18 meeting/presentation, etc.)

[Michael Paoli]

Here's the evaluation feedback and numbers, ... or,
lies, damn lies, and statistics ...
depending upon one's perspective ;-)

First, this is the text that was used for the form - printed two to a
letter sized sheet, and cut in half:

Evaluation Form
____-__-__   ___________________________________________________________
YYYY-MM-DD   Topic/session                                 presenter

Using a scale of 0 (worst) to 10 (best) how would you rate the
following:

Meeting time and location? ___ (able to get to it, find it, etc.)

Food and service? ___ (price/value, quality, service, etc.)

This presentation overall? ___

presentation materials? ___

oral presentation? ___

presenter's knowledge of subject/material? ___

examples/demonstration presented? ___

What do you think were the worst things about this presentation?

What do you think were the best things about this presentation?

Please provide any general comments on how you think this meeting and/or
presentation could be improved or should be changed:

(Optional) Your name and/or contact information:


Certainly feel free to reuse them ... I also made them fairly generic,
so the blanks could be used for other meetings/events.  Hey, maybe at
some point BALUG will be so popular and overrun with potential
presenters, we'll have to thin them out ... and objective data can aid
in comparisons (also helpful for better adjusting one's guesses of what
the general and specific impressions were).

Anyway, first I'll give the items letter labels, so I can have a short
table reference for them (think mnemonics; if the 1st letter isn't
unique, think the 2nd ... follow the bouncing *'s ...):

L Meeting time and *l*ocation? (able to get to it, find it, etc.)
F *F*ood and service? (price/value, quality, service, etc.)
V This presentation o*v*erall?
M presentation *m*aterials?
R o*r*al presentation?
K presenter's *k*nowledge of subject/material?
E *e*xamples/demonstration presented?

And then the numeric data:
  avg. med. low hi. raws
L 9     10  5   10  9,10,10,10,10,5,7,10,10
F 7.9    8  7    9  8,7,9,7,7,8,8,9,8
V 8.2    8  7   10  8,7,8,8,8,8,7,10,10
M 7.3    8  5   10  8,5,8,7,8,5,6,10,9
R 7.4    8  5   10  8,6,8,7,8,5,5,10,10
K 9     10  6   10  10,10,10,9,6,9,7,10,10
E 6.3    7  0   10  7,0,,8,4,5,,10,10

Notes, etc. on the data ... any mathematician/statistician that spots an
error, feel free to correct :-) ...
Some of the forms had empty or non-numeric values for the numeric items,
those are treated as nulls for the sample data (aren't used in
calculating the average, etc.).
Numeric data on the sheets and (further below) text/comments written in
are my most reasonable attempts at interpreting the writing - there may
be at least some ambiguity in the written data (including the numerics).
avg. is arithmetic mean, a.k.a. average (add 'em up, divide by the
number of items)
med. is median - roughly speaking, half are above, and half below (sort
the values, if odd number of items, take the middle one, if even, take
the two next to the middle, and average them) ... anyone, feel free to
correct me if I don't have that done/stated quite right.  (median is
often relatively handy for seeing where most of the values are likely
clustered and/or offsetting small numbers of data points that are far
from the average.  E.g., if I have it right, 1,000 people, 999 of which
have a $10,000 income, and one of which has a $1,000,000,000 income, have
an average income of $1,009,990, and a median income of $10,000 ...
this is why, for example, one commonly hears reports on median home
prices, rather than average home prices).
low and hi. are the lowest and highest values anyone gave.
raws are the raw numeric data
I did check over the raw data fairly carefully (at least to the extent I
could best interpret the written digits), but I haven't double or triple
checked all the other calculations (although I tried to be fairly
careful, and hopefully didn't introduce error(s)).  The reported results
are rounded to the nearest 0.1 (all the numeric responses that were
provided on the sheets gave whole numbers (non-negative integers) within
the requested range).

And, the non-numeric feedback provided.  I did, at least
semi-intentionally, leave enough space around the numeric items so folks
could write some bits of comments in there too, if they felt so inclined
(but didn't explicitly ask for such there) ... anyway, some folks did
write some non-numeric stuff in there (generally along with numeric feedback).

Anyway, here's the non-numeric stuff that was written in, first with the
text they were (or appeared to be) alongside, and then what was written
in, ... less the (optional) contact information ... I also tried to
reproduce the written in text as literally as I could read/interpret it,
and where I felt compelled to squeeze in a clarifying comment, I put it
within angle brackets (<>).  Since The Topic/session wasn't preprinted
on the forms, it's also interesting to note how folks filled that in,
when they did so.

Topic/session <count when 2 or more used same descriptor, and descriptor>
2 SHELL PROGRAMMING <with case variations>
2 Shell Scripting
  Sh Scripting
  Unix Shell
  Unix (sh)
  Oh! sh-- <this one may have been influenced by my describing a cover
sheet I've used in the past for a packet of handout materials on this
topic, where the cover sheet had "KNOW YOUR SH" printed in large block
letters, with the SH aligned on the extreme right, so it rather looks
like it might have gotten cut off on the edge of the page.>

Meeting time and location? ___ (able to get to it, find it, etc.)
V Good

Food and service? ___ (price/value, quality, service, etc.)
God To Exelant

This presentation overall? ___
Good To Exelant

presentation materials? ___
Good

oral presentation? ___
(there wasn't enough time)
fair

presenter's knowledge of subject/material? ___
Good
or so <written immediately after the numeric value>

examples/demonstration presented? ___
- <a literal dash in the field, no numeric value>

What do you think were the worst things about this presentation?
Location at tables was awkward
examples would help
Time too limited - too much background noise, not a lot of visuals
needs more examples, more historical perspective
Hard to hear
location
N/A
the sitting position: cramped not comfortable

What do you think were the best things about this presentation?
thoroughness
Thorough & concise at the same time
relevance to contemporary software
Good knowledge, good pace
Presenter's expertise, + organization of material.
some useful calls and definitions for programming and exec shells.

Please provide any general comments on how you think this meeting and/or
presentation could be improved or should be changed:
Should have more examples
"real life" scripts would show usage and clarify how it all works
Reserve space that's quieter and unseizable; visual examples
Maybe more examples, but you didn't have time.


Anyway, ... I'll comment a (brief?) bit on some of the evaluation data,
both numeric and non-numeric data gathered ...
We were "bumped" from our regular meeting room, and thus had to meet in
the main dining section of the restaurant.  So, no video projection
display, no microphone/PA, other noise (non-BALUG customers) in the same
room, and also didn't want to have our presentation/talk too
loud/disturbing for the restaurant and their other customers present in
the same room.  Additionally, our being "bumped" from the room was
announced, including on the announce list in the evening on the day
before the meeting/presentation - so that might also have scared some
folks off.  Those various factors may have, by their circumstances,
altered how the meeting/presentation was perceived by attendees, and
also altered the crowd (or lack thereof) that showed up for the meeting
and presentation.  There were 15 at the meeting (including the
presenter, myself), 9 evaluation forms were collected (presenter did not
complete a form).  Though meeting circumstances and such weren't highly
similar, there may be enough similarities, and a large enough data
sample set to make useful comparisons between the data that was
gathered in a quite similar manner at the 2005-05-17 meeting.

Other noises in the room, and lack of microphone/PA setup was a fair bit
of an issue, but perhaps not as much of an issue at the 2005-05-17
meeting.  At the 2005-05-17 meeting, we had three tables of people,
roughly 24 or 25 total at the meeting, and the most distant table was a
fair distance from the podium, so at least proportionally, there were
more comments/complaints etc. about lack of microphone at the 2005-05-17
meeting.  At the 2005-10-18 meeting, we managed to squeeze around 2
closely placed tables for the presentation/"talk" portion of the
meeting, so it wasn't so bad - but still could have been significantly
better.  In the written evaluations, at least 2 of the forms received
mention issues with background noise and/or couldn't hear, and there
were also comments during the talk about difficulty hearing the
presenter.

Examples - lack of projector (and microphone) certainly played a
significant part here.  It wasn't feasible to try to do microphone-less
talk over two tables, and also cover showing the screen of a single
ordinary laptop to about 15 people, so, since the talk was the more
crucial part of the presentation, and web materials covering most of the
other material was to be made available a bit later, examples were
mostly (or entirely, depending how one measures) missing from the
presentation/talk at the meeting.  This would also account for the wide
range in evaluation feedback regarding examples/demonstrations
presented.  The written comments typically asked for or suggested
presentation of more examples, and the numeric ratings covered the full
range from 0 to 10, with two null responses on that item (and no null
responses received on the collected forms for any other item which asked
for a numeric response).  It might be viewed that the 0 may have pulled
the average down a fair bit (compared to if it had been a null), but
looking at the difference between the average and median, it may not
have made that much difference.  That one even literally put a dash in
for the numeric rating, is relatively indicative of examples being
mostly missing, or not applicable (none given) - at least at the meeting
itself and the presentation at that time.

The differences in values and sample size may not be large enough in
many cases to be statistically significant, yet comparing to the data
from the 2005-05-17 meeting I note that the data seems to indicate or
hint that:
meeting time and location is quite good for folks (but again, we may
have the "problem" of the self-selecting sample set - those for whom
it's not good don't show up to fill out the evaluation form),
food/service may be slightly better in main dining area, but ratings
are close enough, difference may not be statistically significant,
folks liked the LVM presentation a bit more than the shell presentation,
folks liked the LVM materials somewhat more than the shell materials,
folks liked the oral delivery of the LVM talk somewhat more than the
oral delivery of the shell talk,
folks significantly preferred the LVM examples/demonstration to the (lack
of) examples/demonstration on shell.
Its difficult to estimate accurately how being bumped from our "regular"
room into the shared dining area may have altered these various outcomes.

Not that it would make a huge difference, but also, due to our being
bumped, I didn't have the set-up time I otherwise would have expected to
have, and also the "announcements" business bit of our meeting went on a
bit into some discussion - so between those two items I probably lost
about another 10 minutes of presentation time - something that can be,
and may have been fairly significant - particularly with a presentation
that's already a fair bit of a squeeze to do in ~75 minutes (I think I
did it in <~=70 minutes - I'd targeted my timeline more towards a
minimum of about 75 minutes, and hopefully a moderate bit more
(particularly more Q&A and mini-demos and examples at/towards the end).
With the physical arrangements and no projection screen, and somewhat
more delayed start and no set-up time, the tail end was mostly just a
bit of Q&A, without really getting to show examples or any
mini-demo(s)).

Well, ... enough commentary/analysis on evaluation data ... at least at
the moment and from me.

References/excerpts:

similar data collection and analysis from 2005-05-17 meeting:
http://lists.balug.org/pipermail/balug-admin-balug.org/2005-May/000032.html
2005-10-18 meeting, announcement, reminder announcement, and "oops we're
bumped" announcements, respectively:
http://lists.balug.org/pipermail/balug-announce-balug.org/2005-September/000047.html
http://lists.balug.org/pipermail/balug-announce-balug.org/2005-October/000048.html
http://lists.balug.org/pipermail/balug-announce-balug.org/2005-October/000049.html