The Bird's Eye View

Hi, everyone!

To make it easy for you to decide what you might like to examine more closely in this blog, I offer the following "bird's eye view." There were many more sessions and these posts are just a summary of the ones I attended. Some posts contain links to resources. Posts are in reverse sequence, so if you want to journey from the beginning, you'd start at the last entry.

All the best,

Ludy Goodson goodsonl@erau.edu
Senior Instructional Designer Course Design & Production - Worldwide Online Embry-Riddle Aeronautical University 600 Clyde Morris Boulevard Daytona Beach, FL 32114-3900
Phone 386.947.5210 Fax 386.226.7107
goodsonl@erau.edu

Online Homework…
Online homework gave students practice and quick feedback. Students felt the homework helped them to learn, but they would not have done homework if it had been ungraded.

Southeastern SoTL Colloquy
This regional group is just beginning and this post is a brief overview of our “brainstorming” of ideas for the future.

Interactive StudyMateTM…
Surveys showed that students used and valued a variety of electronic study aids (ESAs).

Group Problem Solving…
How can you create a group structure for effective learning with both individual and group responsibility? Take a look at the recitation structure – it’s a good model for planning both classroom and online group process.

Guided Inquiry…
How should you design differently for beginning vs. advanced students? This study uses scholarship to answer the question with results that showed both learning and appeal to students.

The Scholarship of Teaching…
Here you will find encyclopedic links to resources for support on developing and using the scholarship of teaching and learning.

Welcome to the Dance…
Several major learning taxonomies differ in language and architecture. If you want a “Cliff notes” edition of each taxonomy. this is the blog for you. If you want more details, contact goodsonl@erau.edu.

Scholarly Teaching and Scholarship
Scholarly teaching uses the results of scholarship. Scholarship is not just another study; it needs to beplaced in the context of the body of knowledge.

The Power of Course Design
The choice of “significant learning” and Fink’s design framework operated to improve the value of learning and its coherency in an online course. For more details, contact goodsonl@erau.edu or iahn@georgiasouthern.edu.

Online versus In-Class Teaching…
Assessments were grouped by level of learning outcome; results on a final exam were compared to learning styles. Students at either extreme of learning style performed better than those with “balanced learning styles.”

Student Self-Perceptions…
Use of a confidence-rating system helped students; students attribute value to the confidence-rating the system; grades improved; grades dramatically improve with the addition of a pretest. For more details, contact goodsonl@erau.edu or dslater@georgiasouthern.edu.

The Praxis of SoTL
SoTL is reflective, reflexive, and recursive.

Community, Voices, and Portals of Engagement
The intersection of communities, learning, and technology offer the opportunity for global perspectives and more open inquiry in a digital community.

Improving Academic Quality…
“Ramping up” a culture of assessment requires faculty participation and activities, including instructional design, faculty development, stipends, “coaches,” and a hefty budget.

A Methodology for Developing…
Rigorous validity and reliability procedures led to the development of a rubric that improved student performance and assessment of learning.

SoTL Commons Reception
View some of the beautiful people and get a feel for the international community at this conference.

Online Homework: Student Attitudes and Learning Outcomes in a General Economics Courses

Large classes offer a major challenge to giving guidance and feedback to students. Online homework assignments with automated feedback offer potential for student practice and guidance. In the study of such homework in Economics and Business Statistics courses, there was an advantage not only for multiple choice items, but but essays which become easier to grade when posted online (no need to deal with handwriting fog!).

Tools for applications discussed in this session included MyEconLab, Respondus, and Aplia in the Moodle LMS. With the homework system, students often also have the advantage of teh e-text, which tends to be cheaper than the print textbook.

Student surveys showed that most felt that they learned more from online homework, and some saw no particular difference with homework online; most felt that it took less time or that there was no difference in the time required to do online homework. The Pearson correlation coefficient did not show a strong correlation of homework grades with the final exam scores. Additional analysis needs to be done to complete the study.

Even though students reported that the homework helped them to learn, they reported that they probably would not do the homework if it were not graded.


Presenter: David Doorn, University of Minnesota Duluth

Southeastern SoTL Colloquy

Our planning session began with introductions of the Founders Council members. The items for discussion covered the bylaws and constitution, opportunities that we want to offer through the Southeastern SoTL Colloquy, affiliations that we would like to establish and sustain with other organizations such as CASTL, ISSOTL, and SoTL Commons. We briefly discussed other operational issues such as dues and the possibility of institutional memberships.

We spent most time on discussion of the goals and purposes of the organization and how to implement them. We created a "brainstorming list" on implementation avenues such as campus visits, individual collaborations (I invited everyone to come to the beach!), retreats, and workshops; further development of the Web site, development and sharing of "white papers" on hot topics such as promotion and tenure, IRB processes; and what kinds of files may be public vs. password protected.

In this discussion, we flagged some high vs. low priorities and will proceed with further follow-up on these issues after the conference.

Discussants: Southeastern SoTL Colloquy Founders Council Members who were attending SoTL Commons Conference

Interactive StudyMate TM-Generated Electronic Study Aids: Do Students Find Them Useful as Learning Resources?

The presentation began with a review of the options made available with the use of StudyMate 2.0, and a review of examples (flash cards, matching, crosswords, fill-in-the-blank, glossary, quiz, and challenge game). Use of this tool offers more elements of interactivity and feedback. Images and other media can be integrated.

The primary motivation for investing development in the StudyMate activities was to get students to participate. Large numbers of students were doing poorly, were not engaged in lectures, and would not come to office hours. If students would not interact with the course content, they would be unlikely to succeed in the course.

The electronic study aids were made available for freshmen to use in 8 sections of a beginning course taught by 5 instructors. A survey was given at the end of the course to determine whether students had used the materials, when they used them, and their perceptions of their value to their learning. Results indicated high use, high ratings for their effectiveness, and use at all hours except 3 am to 6 am and more late at night. Students had a high preference for feedback (96%) with the ESAs and would prefer a Web link with more quality information with the feedback (87%). If ESAs could be loaded on to iPods or other portable players, about 60% would probably download and use them.

There was no comparison to measures of learning outcomes (grades).

Presenter: David Kreller, Georgia Southern University

Group Problem Solving in General Chemistry Recitation to Promote Learning

The context of learning in the general chemistry course was a large number of students in each recitation section (e.g., 90) and contact-hour limits placed on faculty time. The challenge was to determine how to reduce the numbers in the recitation without increasing contact time. The solution was to increase the size in the lectures and increase the number of recitation sections.

Then consideration was given to the recitation structure, and key features included:

- mixed skill levels so students would learn from each other (Math SAT scores gave ability levels)
- requirement of student preparation for recitation through homework assignments
- teaching assistants for recitation and grading (undergraduate)
student participation and peer evaluation

WebAssign was used for online homework. The problems created were algorithmic, so students couldn’t mindlessly copy from each other.

In groups, students were given multi-step problems; answers were not obtainable from direct application of a formula, problems that encouraged discussion of ideas (estimation, qualitative answers).

Using a two-sample t-test to compare averages and a Mann-Whitney test to compare medians, results showed that test score averages before and after the group problem solving methodology, were not significantly different. But there was a 10% increase in A, B, C, and D grades that could not be attributed to differences in students’ SAT scores.

The ERIC Document Service Reproduction No. EJ814870 can be used in a search for the paper.

Presenters: Madhu Mahalingam, Fred Schaefer, and Elisabeth Morlino, University of the Sciences in Philadelphia

Guided Inquiry in an Upper Level vs. Lower Level Undergraduate Course

This session began with a review of the shift from “teaching is telling” to “teaching is enabling,” from “knowledge is facts” to “knowledge is understanding,” and from “learning is recall” to “learning is an active reconstruction of subject matter.”

With this paradigm shift, guided inquiry with learning through constructivism would seem to be good fit. Recognizing differences in students’ levels of expertise in a beginning vs. advanced chemistry course, however, demands some analysis of levels of inquiry that are appropriate.

In general, beginning students without prior knowledge of chemistry cannot start with open and unguided inquiry, but this does not mean that the lecture format is the alternative. Instead, two other options work: structured lecture-interactive and guided discovery.

Laura reviewed some key concepts from references on learning:

- the perception-input-filtering-storage-retrieval process of science learning - Johnston, 1998)
- the exploration-concept-invention the science learning cycle (Karplus, 1967; Piaget, 1964)
- key ideas about learning – constructing to understand, exploration-concept formation-application cycle, connecting and visualizing concepts and representations, discussing and interacting with others, and reflecting on progress and assessing performance (Bransford, et al)
- learning levels – information (recall), concept (classifies), simple rules (demonstrates), and higher complex rules or problem solving (generate) – (Gagné & Briggs, 1974)

Next, she considered what level of learning is expected in the introductory vs. the upper level course. The Process Oriented Guided Inquiry (POGIL) model was used for the introductory course and a team-based learning model (Michaelson) was used for the advanced course.

In both models, group work was done in class. In POGIL (cooperative learning) the groups changed, there were 4 or fewer in each group, roles were assigned, process skills were critical, group work might be graded, prompt feedback would occur sometimes, and peer assessment would occur sometimes. In team-based learning, the groups stay the same the whole term, with 5-7 members, no roles are assigned, low-concern for process skills, group work is graded, prompt feedback is provided, and peer assessment is critical.

Results for the introductory course of moving from the lecture-interactive to the inquiry teaching showed a higher distribution of A’s, B’s, and C’s with lower DWFs. For the upper level course, the shift from lecture-interactive to inquiry resulted in lower DFWs (small improvements in grade distribution, but too small to draw conclusions yet).

In the introductory course, analysis of questions grouped by learning level shows more gains (significantly higher) for recall and conceptual learning. In the upper level course, there were no clear differences across types of learning outcomes. In both courses, the average difference in correct responses to common items across exams was persistently higher in the inquiry classes.

Student evaluations of guided inquiry in the introductory course revealed some benefits (socialization, staying on task, and help in problem solving) and some problems (rudeness, feeling of inadequacy, group lack of knowledge or too much socialization, insufficient explanation and guidance).

Student evaluations of inquiry learning in the advanced course gave high marks for the class activities, instructional approach, and course resources.

Presenter: Laura DeLong Frost, Georgia Southern University

The Scholarship of Teaching: What's the Problem?

My friends Laura DeLong Frost and Don Slater and I caught up on research ideas while eating a high-carb lunch of sandwiches, pasta, potato chips, and apple pie with buttery flaky crust. By the time our keynote speaker began, we were starting to feel the effects and I just wanted to nap. So, if I missed anything critical, you will know why.

Randy Bass wrote The Scholarship of Teaching and Learning: What’s the Problem? back in 1999. He mentioned from that artilcle, “the culmination of nearly a decade of discussion that began with the 1990 publication of Scholarship Reconsidered (Boyer), and then refined later in Scholarship Reassessed (Glassick, Huber, Maeroff, 1997)” and drew attention to the theme that we have “ideas worth spreading,” playing a clip from TED talks by Sir Ken Robinson. This 18-minute video provides a perspective on the purpose of education.

Randy emphasized the need for communal work in the intersection of teaching, learning, and technology, and the need for the visible knowledge of scholarship of the teaching and learning process, referencing the Scholarship of Teaching and Learning Commons, The Visible Knowledge Project, and abundant other resources available at the Gallery of Teaching & Learning.

Randy reminded us of the original value of the educational system, designed to meet the needs of industrialization, and how often students have been benignly told to take courses that will lead to a job, as “you’ll never get a job doing that” in which “that” represents what the student was really good at doing and enjoyed. He thinks our education system has “mined our minds” for a particular academic commodity, and that the paradigm shift from “providing instruction” to “producing learning” back in 1995, has made progress. The concern by both Robinson and Bass is the loss of creativity.

Barr and Tagg in their 1995 publication of From Teaching to Learning – A New Paradigm for Undergraduate Education sparked a movement. Yet, looking at one example of progress since then, he pointed out that the “high impact activities” of NSSSE include activities such as internships and extra-curricular activities, leading him to wonder, “So, what are the low-impact activities?” "Would those be our courses?!" Students are learning even if we are not teaching. What progress have we really made?

The Carnegie Foundation also provides resources at Strengthening Pre-Collegiate Education in Community Colleges (SPECC). From this project, Randy played a clip from The Think Aloud project to illustrate the difficulty and complexity of the learning process even for basic skills (select “Think Aloud with Jay’s Commentary” to view the clip). Other sample cases on basic skills are available at Windows for Learning: Resources for Basic Skills Education.

Randy elaborated on the meaning of The Middle of Open Spaces: Generating Knowledge and Learning through Multiple Layers of Open Teaching Communities by Bass and Bernstein (or see the chapter among a list of related articles). These “middle spaces” between practice and published literature are “informal and not tied to high stakes.” In addition to the Visible Knowledge Project, Randy encouraged use of the KEEP Toolkit available for developing Web sites with visually appealing representations.

The Carnegie Foundation for the Advancement of Teaching has valuable resources to support scholarly teaching and the development of scholarship at CASTL Higher Education. Another resource is Collaboration across Disciplines on a Digital Gallery. As an example of collaborative inquiry, Randy referenced the “Lesson Study Project” at the University of Wisconsin, La Cross (select Makoto video, if interested in viewing the interview about lesson study research) and focused on Lesson Study: An Experience in Collaborative Inquiry.

Randy asked “How would you change your study to ask your question collaboratively?” and “Who else is asking some version of the same question?” And, he directed us to the Faculty Inquiry Toolkit.

He next drew attention to The Gap between Ability and Sustainability by Hern who had noted “The gap between students' ability to perform and the performance they actually sustain over the semester. Chronic condition in community colleges whereby students earn passing grades on individual assignments then withdraw or fail class.”

He mentioned digital stories as an emerging “signature pedagogy” of new ways of communicating as in multimedia authoring such as student posters and digital stories of student learning like those at gnovis journal (the example shown at the keynote was of a reflection on the role of makeup in a coed’s daily routines).

Randy emphasizied that we must deal not with just the “how’s” and “what’s” but the “why’s” of what we do. And, we need to move from the “knowledgeable” to “knowledge-able,” including new models of assessment such as those described by Wesch in From Knowledgeable to Knowledge-able: Learning in New Media Environments.

Keynote speaker: Randy Bass, Georgetown University

Welcome to the Dance: Partnering Up Taxonomies and Research

In theory, authors of major learning taxonomies used the same body of work to classify learning outcomes, yet they have wide variations in terminology and classifications.

The session began with group information about four taxonomies and brief dialogue about their origins: Bloom and “new Bloom,” Webb, Marzano, and Gagné and Briggs. Most were familiar with Bloom, unfamiliar with “new Bloom;” one was familiar with and using Webb; one was familiar with and using Gagné and Briggs; none were familiar with Marzano.

The purpose and architecture of each taxonomy was reviewed, after which one participant aptly described the differences in terms of producing "cognitive dissonance" and another suggested that the taxonomies simply could not be integrated.

Bloom: The committee of educational psychologists that developed the Bloom’s taxonomy did so for the purpose of improving assessment; the “revised Bloom” shifts the focus to learning processes that can then be applied to different types of knowledge; for example, one can “remember procedures,” “understand procedures,” “apply procedures,” “analyze procedures,” “evaluate procedures,” and “create procedures” (evaluate is not the top level, but now second from the top). There were three domains: cognitive (remember, understand, etc.), psychomotor (perform or execute…), and affective (develop a commitment to a set of behaviors). Forehand provides a “Cliff Notes on Bloom’s Taxonomy.”

Webb: Webb’s model originated to assess the alignment of test items with targeted learning levels in curriculum standards. Webb developed a “depth of knowledge” (DOK) taxonomy with four levels: (1) recall; (2) skill or concept, (3) strategic thinking, and (4) extended thinking. (Webb’s DOK is one of six dimensions in determination of alignment.) Webb’s DOK has been used with at least ten states and multiple subject areas. The Wisconsin Center for Education Research describes the DOK levels and the alignment process. This model does not have a separate classification scheme for the affective or psychomotor domain.

Marzano: In 1988, Marzano’s framework had five dimensions of thinking: (1) metacognition, (2) critical and creative thinking, (3) thinking processes, (4) core thinking skills, and (5) connecting thinking to content. Since that time, the Mid-continent Research for Education and Learning (McREL), with Marzano as CEO, has reframed these dimensions: (1) attitudes and perceptions, (2) acquire and integrate knowledge, (3) extend and refine knowledge, (4) use knowledge meaningfully, and (5) productive habits of mind. This approach integrates affective with cognitive domains of learning. It does not explicitly address the psychomotor domain.

Gagné & Briggs: Kowch at the University of Calgary provides a summary of the “varieties of learning” and theory background. The main varieties are: (1) intellectual skills, (2) motor skills, (3) verbal information, (4) cognitive strategy (how to learn), and (5) attitudes. Intellectual skills develop in a hierarchy and include: (1) discriminations, (2) concrete concept, (3) defined concept, (4) rule, (5) higher order rule (include metacognitive and complex problem solving here).

These taxonomies evolved from analysis of educational research, not from the research per se. For example, although Gagné and Briggs often conducted or used educational research in teaching and training applications, Gagné’s development of the taxonomy evolved from years of analysis of learning into its categories. And, Bloom, the typology with which most are familiar evolved over an eight year period from the analysis of a group of psychologists. As another example, Marzano and others have published classroom instructional strategies that work. But the “dimensions of learning” don’t align directly with the categories of “what works” in the classroom.

In small groups, session participants tackled the question of whether the taxonomies bear a reasonable relationship to each other; one of the groups saw a clear connection of Webb to several parts of Bloom, another saw some relationship of Gagné & Briggs to some of Bloom, and another found some relationship of Marzano to some of Bloom.

Another question was whether the educational studies that have been conducted since before Bloom, can or should inform a stronger and more coherent description of learning outcomes? And, if so, what would be the value of doing so?

Session participants were provided with a summary of educational research for the past twenty years organized into different types of learning outcomes to show the types of instructional and assessment strategies that fit with each, as an example of one approach to integrating the research with the types of learning outcomes.

After the session, two of the participants wanted to know why I considered Dee Fink's approach as a "course design framework" rather than a taxonomy for learning. They suggested that Dee Fink's classification should be considered a major taxonomy, and one of them plans to formulate the logic for this consideration and send forth her notes, with the possibility of our future collaboration on this analysis.

Presenter: Ludwika Goodson, Embry-Riddle Aeronautical University of Science and Technology

Scholarly Teaching and Scholarship

What do you value in art? Is it realism? impressionistic that you prefer? and does this align with your discipline? There is Van Gogh’s Potato Eaters and Picasso’s cubism. These topics flavored our lunchtime conversation before Laurie Richlin’s keynote presentation.

Laurie began by asking who is Charles Drew? He invented a system for saving blood plasma (blood bank) The Charles Drew University of Medicine and Science, a teaching-based hospital and medical university, was named in his honor. In addition to her work at the university, Laurie is executive editor of the Journal on Excellence in College Teaching. She invites strong evidence-based articles about the scholarly approach to teaching and learning and the advancement of scholarship in this field. She mentioned the Lilly Conference on College and University Teaching as another community of learning.



Some key points:

- learning outcomes (LOs) are "the center of all good"
- knowing “what is learning?” is a foundation for what we do
- scholarly teaching is not the same as scholarship of teaching and learning
- what are the answers to: What is evidence? Why don’t faculty use it? Why don’t faculty produce it? What are the qualities of evidence?

Teaching without learning is just talk. We may “teach” without anyone learning what we teach. Learning is a physical change; it establishes pathways and connections. In the past teachers talked too often about what they did and not what the students did. (Steadman and Svinicki provide a summary of how Angelo’s and Cross’s classroom assessment techniques, 1993, provide a “Gateway to Better Learning”).

The “scholarly approach” is the investigation of the literature which then directs evidence-based course design and teaching, while “scholarship” is the development of such evidence. Such scholarship requires a systematic evaluation, and ideally it will be placed into the context of the whole knowledge base (not just a pebble on the ground).

Keynote speaker: Laurie Richlin

The Power of Course Design

Goodson and Ahn reviewed the threads of design elements that make a variety of support technologies effective for learning, including instructional television, audio, computer applications, and particularly online learning. Russell after a review of over 355 studies about online vs. classroom, concluded that the power was in the design. Recent research suggests the same (Zhao, Let, Yan, & Tan, 2005) where advantages of online seemed due to differences in design, not in technology. Fink's recent work (2007) also shows the power of design in several studies that produced improved learning through a design process that aligns goals with activities and assessments and feedback for "significant learning" outcomes.



In this session, the case study of the design of an aesthetics course was analyzed to show the progression of decisions and the evolving architecture for the course content, the learning activities (including quizzes), and the assessments. The first stage was the question of what constitutes significant learning? The former course design focused on fashion for body types in restrictive stereotypes; the new course design focused on authentic design processes that lead to fashion design (not just choosing fashion), beginning with sources of inspiration and leading to analysis of fashion elements such as line, balance, rhythm, complexity, and color. In this process of deciding on "significant learning," the human dimension was deepened, and a sense of caring was embedded in the images and language structure of the course.

Shifting to meaningful "significant learning" was beginning with the "why" of what we teach. This then triggered decisions about the course goals, leading to the major modules in the course. Quizzes, discussions, and assessments were designed to match each other and required responses from students that often did not have a "yes" or "no" or distinctively correct answer, but required the application of knowledge and principles.

Feedback was staged to allow students to take the quizzes with automated feedback, in the form of correct answers for multiple choice questions, and in the form of "Did you cover...? If not, you may need to..." for more "open" questions.

Discussion questions asked for examples of concepts and reasoning; the process was to allow discussion without response from the instructor until near the end of the week, who would then summarize the main ideas from the discussion. The instructor also would intervene if a concept was not being addressed correctly so that students would not go down a dead end.

In this session, the group reviewed several module structures to follow the design and architecture of the "information and question pages" and they listed and viewed a Camtasia component of instruction, noting the explicit character of the steps and the use of audio to give directions.

Goodson and Ahn returned to the simplicity of the concepts of design, and the power they possess in transforming the learning - first, determining what is significant learning, then aligning all components of the course with this decision.

Results in this case study as compared with the older edition of the course included: significant learning and insights expressed by students in discussions about "the aesthetics of fashion design" rather than "body type," a "world of fashion" rather than restricted stereotypes or parochial fashions, authentic images inclusive of multiple cultures and both male and female gender rather than females only, a larger distribution of higher grades, and no failures.

Presenters: Ludwika A. Goodson, Embry-Riddle Aeronautical University, InSook Ahn, Georgia Southern University

Online Versus In-class Teaching: Learning Levels Explain Student Performance! AND Are Some Students Better Suited for Online Class...?

I'm reporting on two sessions in this post because they are tightly interwoven and created by the same presenters.

Both sessions began by reviewing the growth rate in online learning (Sloan Report, 2007). In one, we responded to the question "Is online learning for everyone?" Some answers focused on convenience and flexibility (Who is Best Suited for Online Learning? Online Universities Weblog). Some focused on traits commonly cited such as independence, discipline, self-motivation, liking to read and write, ability to stay on task (Is Online Learning for You?). The group agreed that students with these traits would be successful whether learning online or in the classroom.

As to whether there are differences in learning from online vs. onground delivery, both sessions referred to findings in some studies that online is better, in others that classroom is better, and in many that there are no differences (as reported at WCET No Significant Differences Web site). (In this review of findings, there was no analysis offered of whether course design or instructional activities could be the reason for divergent results more than mode of delivery. However, participants in both sessions brought up the question of the alignment with instructional strategies.)

The focus for each session was whether students with different learning styles may perform better online (Index of Learning Styles Questionnaire by Soloman and Felder). To measure student learning performance in this study, the exam had items explicitly aligned with each level of the "old Bloom" taxonomy - 5 items for each level of learning. The exam creators checked reliability of their classifications of items into each of the learning categories.

The course, taught online and onground, had a common syllabus, common final exam, comparable number of students, and students with comparable GPAs. These were some of the differences:

• there were fewer students whose majors were quantitative (e.g., computer science) vs. qualitative fields (e.g., liberal arts) in the online delivery
• there was one instructor for the online course, and multiple instructors for the onground delivery
• the instructional activities and quizzes during the course were idiosyncratic to the preferences of the TA's, thereby making the instruction different in each section of the course

Using classic inferential statistical analysis, the study seems to show that extremes at either end of some learning styles, e.g., "active-reflective," do better than "balanced" learners in an online class.

While there seemed to be no differences in online vs. onground learning except in synthesis outcomes, the structure of the assessment design in this study holds promise for sharper analysis of impact on learning outcomes. The data produced some controversial speculation about whether there will be NSDs at all but the synthesis level; discussion in each session pointed out a couple of possible pitfalls in analysis - namely, if the course has not been designed and delivered with synthesis learning activities in the online as well as onground course sections, then it is not valid to conclude that either mode of delivery has produced better results than the other.

The data showed, after controlling for GPA and differences in qualitative majors, that online students did not perform as well on the synthesis items. But the study did not show the reasons.

The design also omitted the "evaluation" level of learning, so performance on that level was not tested.

The paper on this research should be developed in about a month, and I've requested a copy of the complete report from each of the presenters in each session. One of the presenters expressed interest in finding out more about instructional strategies known to work for different types of learning outcomes, about Webb's curriculum and assessment alignment model, the hierarchy of Gagne and Briggs, and re-examination of what is now defined as the highest level of learning in the "revised Bloom's taxonomy."

I think this was a well designed study and could serve as a good model for future exploration.

Presenter: Milind Shrikhande and Richard Felder Georgia State
University

Student Self-Perceptions and Cognitive Abilities

Do students’ self-perceptions influence their cognitive abilities, and how do we know? In this quasi-experimental study, we examine possible answers to this question. Some of the concepts relate to early work by Bandura and models that followed on how self-efficacy influences learning performance.


In the session's discussion about self-perceptions, many expressed a belief that self-perceptions influence learning, but only one had gathered statistics to confirm the belief. Several assumed that low self-perception would produce low performance, but after our session, reported that their thinking had now shifted; indeed, in some contexts low self-perception may lead to less persistence, but in others such as the one we provided, it may lead to greater persistence. We also had a poster session on this same research, and discussion was fairly nonstop during this time.

We looked at self-confidence before and after instruction as one component of self-perception. A confidence survey was created for each module of a course in which students rated their confidence in performing discrete tasks and objectives.

In 2007 and 2008, pre-instruction confidence ratings were persistently low for all units and tasks in the course while post-instruction confidence ratings were persistently high. Use of the confidence rating system seems to improve learning as measured by grades, when compared to deliveries of the same course that did not use such a system (2006).

In 2007 and 2008 in courses that used the confidence rating system, students reported that the confidence rating process was of high value, and this seems to be associated with all grades given in the course. We think that these ratings suggest higher student engagement with the learning tasks.

In 2008, a pretest was added as a measure of pre-knowledge, and scores were persistently low (no score above 8). In this course (multiple sections), the pretest combined with the confidence rating system seems to have produced dramatic improvements in grades.

Anecdotally, students reported to the instructor that they were informed of what to study in the confidence surveys, that some would copy the surveys and expand them to use them as study guides. The combination of the pretest with the confidence rating provided them with greater clarity about expectations.

The next stage of work for this study will be systematic statistical analysis of the data. We invited advice from anyone in the group who could recommend statistical analyses that would fit with our context. One person took detailed notes about our project and will be sending us follow up recommendations about how to proceed with more rigorous statistical analysis.

Presenters: Don Slater, Georgia Southern University & Ludwika Goodson, Embry-Riddle Aeronautical University

The Praxis of SoTL

The scholarship of teaching and learning (SoTL) is reflective, reflexive, and recursive.

• Reflective: SoTL requires contemplation (analysis) of context and variables.
• Reflexive: SoTL requires ethical and moral thinking – who would benefit from the study? in what ways? who would not benefit?
• Recursive: SoTL comes full circle as the context shifts (unlike controlled studies)

Discussion during and after this session focused on challenges and rigors of the qualitative method, the value of reflecting on process and publishing in journals on such processes.

Brian, who said he had been a “classical statistics” person, explained that he had shifted perspective to seeing the value in qualitative studies. He mentioned that the process and product of Einstein’s thinking would not have been considered as worthy of publication by many journals. Others are beginning to realize the importance of that type of scholarship, too.

There are standards that fit well with the study of what works in authentic contexts of teaching and learning. The group seemed to conclude that the standards of research in SoTL need to be communicated and met as we look at the evidence of what works in the teaching and learning process.

Presenter: Lorraine Gilpin, Georgia Southern University

Reference: Gilpin, L. (2007). Unearthing the scholarship of teaching and learning in self and practice. International Journal for the Scholarship of Teaching and Learning, 1(2), 125-135.

Community, Voices, and Portals of Engagement

Our luncheon keynote speaker probably was thinking globally, while a biochemist Margie Paz, originally from the Phillipines, and now at Griffin, Georgia, talked with me, and we discovered that we both had attended Iowa State University in Ames, Iowa. And, we shared some stories about Ames, Des Moines, Griffin, and the Phillipines.

Our conversation was good prep for Dr. T’s fresh perspective on Yo Yo Ma’s “Silk Road Ensemble” and the dimensions of learning in that project. I was struck by the concept that a body of knowledge is unknowingly privileged by particular voices. Dr. T. showed us how threads of perception of the same event in historical images provide different knowledge about that event, and how students can be engaged in discovering, in a digital community, these different types of knowledge. She showed us the project “Perry Visits Japan.”

Dr. T focused on the hybrid of cognitive and sensory frameworks to transform comprehensive and critical analysis, and how these come to define “signature” pedagogies, such as “grand rounds” in medical teaching.

As I write these notes a day later, two teachers across the hallway are discussing spiritual understanding and personality in the teaching process. This is not unlike the concept that Dr. T. mentioned of designing “open-room for debate and interpretations” into the assignments that we create for students. She reflected on the challenge for many teachers of planning inquiry with open-ended studies and absence of conclusions into a study process.

In this context, layering communities and voices into portals of engagement led to the creation of “research teams” whose members came from different countries, who were provided with real data about a pervasive real-world health problem, and were invited to decide what to do with the data. This approach was much less prescriptive than many learning assignments. The teams had to
decide what was important to them in their personal lives, what they care about most, and then could proceed with deciding what to do about the data.

Dr. T. described the distance learning design-studio concept. She added questions about what to visualize from what perspective? What to scaffold? How to develop productive habits of mind? Reflection on McLuhan’s (1961) The Gutenberg Galaxy notion of extending our senses outside of us with the social world, and how this is like what happens when a new note is added to a melody. And what biases do we bring to the curriculum?

Keynote Speaker: Dr. Kathy Takayama

Improving Academic Quality through Outcomes Assessment and Active Learning Strategies - a Model for Effective Institutional Change

To kickoff this session, we had a case study and a chance to talk to each other about approaches to "ramp up" the culture of assessment on campus. Several of us shared common experiences with resistance from faculty in shaping up outcome statments, but we also shared ideas about how to support faculty in "driving" the "ramp up" process. These included building faculty learning communities in each discipline with a common literature base, gaining administrative support for the assessment culture, and scheduling roundtales to define learning outcomes.

In this project, goals were focused on increased use of outcomes based assessment and active learning in courses. These goals were accomplished through activities of developing learning outcomes, curriculum maps, and competency growth plans within programs; course embedded assessments, direct and indirect assessment methods, and criteria developed in rubrics; instructional design support for outcome achievement, and focused faculty development programs with support such as stipends, money for conference travel, "faculty coaches," and a resource library and Web site.

In addition to sustained incentives for faculty participation, instructional design support, and "faculty coaches," there also were demonstration fairs and student participation in NSSE was at a remarkable 55% response rate, above the 25% average. Presenters consider this effect a benefit of the increased active learning and better alignment of assessments with learning outcomes.

The budget for this five-year project was $1.78 million with support from USDE Aid.

Presenters: Mary C. Hassinger & Theresa R. Moore, Stephens College & Viterbo University

A Methodology for Developing and Validating an Assessment Rubric

The techniques in this study are commonly expected in the process of developing a legally defensible certification exam. The presentation, however, focused on exceptionally rigorous application of these techniques for assessment in a context of teaching.

With the goal of improving a rubric's validity and reliability, the process in this study began with development of learning objectives and a review of related professional guidelines and literature before engaging in academic discussions about the constructs in the rubric. Representatives from the academic area and from the profession were engaged in discussions about criteria and weights for the rubric. A process of consensus moderation known as the Delphi technique was used to reach agreement among review groups, thereby reducing variance in scoring.

Among the reported results, the research team reported the observation that there was more agreement in using the rubric with an "A" paper, but below that level, variance becomes greater as teachers consider how to give feedback without discouraging the lower-performing students. The questions after this observation: What is fair? Don't students deserve equitable evaluation? Or do we want to encourage a system in which students "section shop?"

The rubric in this study was a "News Release Evaluation Rubric" had a left column with five major constructs, each with a checklist of elements, and a right column with a % weight for each major construct.

Presenters: Sandra Allen, Columbia College Chicago and John "Ed" Knight, University of Tennessee at Martin.

Provost's Welcome and "Sweet Iced Tea"

This morning GSU Provost Linda Bleicken told of years past in which she had paid her own way and stayed in college dorms just to be able to go to conferences on teaching and learning. While some did not understand, she knew then and now of the value of examining the evidence of what works in the classroom. On a light note, she mentioned how much Georgians like their "sweet iced tea," and at last year's conference, visitors from England explained they would prefer to drink it hot, particularly in the morning! She encouraged all to feel welcome and let the team know of what will give us comfort as we weave the thread of teaching and learning in our journey through this week's conference sessions.

SoTL Commons Conference Reception

Today was the kickoff for The SoTL Commons: A Conference for the Scholarship of Teaching & Learning. The welcome came with warm hugs from Patricia Hendrix and Steve Bonham (former colleagues at GSU), Stacy Kluge (conference coordinator), and Alan Altany (creative scholar and spirit of this event). Way to go!

Folks are here from Australia, Canada, China, Ecuador, Jamaica, Malaysia, Nigeria, Singapore, South Africa, Taiwan, Turkey, United Arab Emirates, United Kingdom, and the United States. Two of the people from South Africa joined us at our table with global energy that would be hard to imagine. They were vibrant and enthusiastic with lots of questions even after 20 hours of flying time. And, they were beautiful!

It was great seeing some of my old friends, Linda Mullen and Trent Maurer, both teachers at GSU (Trent is now a happy newlywed!), Lugenia Dixon, who's a writer and teacher from Bainbridge who'll be reporting on research about the effects of self-talk on success (I'll find out more on Thursday), and my friend Don Slater, from GSU who's presenting with me in a session tomorrow about the influence of self-perception on performance.

At this evening's reception, we had exquisite musicians (Jean-Paul and Dominique Carton, and David Posner), fine food, warm collegiality, and international flair! We were all taking photos and I'll share some later.

For this week, we have a 72-page program and the next three days start at 7:30 AM, the first two going until 7:30 PM. Some choices will be difficult and wonderful!