Thursday, March 15, 2012

Review of Open Assessment Topic

I reviewed the materials provided on the topic link for open assessment.  I reviewed all of the materials provided (OK, I did not watch the full two hours of the first video) and found myself trying to relate the concepts of badges to the paradigm of the world in which I live.  I have made a formal study of effective assessment both as part of my academic endeavors and as part of my current professional assignment.  I am also closely engaged in our regular accreditation reviews at the University.

I was trying to reconcile this concept of open badges (particularly the different assessment approaches) with the recent emergence of more restrictive and formal assessment efforts proscribed for accreditation standards, which also seem to be the prevailing view of governmental thinking about accountability.  It seems that a rigorous outcomes-oriented approach is the foundation for every new proposed evaluation of effective education, at least here in the United States.  Not only are these outcomes-based approaches being centered in accountability and accreditation, there are also movements for instructional objectives, and assessments, to be common to all institutions.  While these trends have become the norm in K-12 public education, they are bleeding over to higher education, particularly in the accreditation process.

The thing I wondered as I read through the white paper at the Mozilla badge wiki was how the varied approaches to assessment and recognition for achievement (or learning, if you will) through the issuing of badges aligns with the current emphasis on formalized, standardized, outcomes-based evaluation of education in accreditation and accountability processes.  The white paper notes that the badge concept is an appropriate response to the change in learning brought about by the Web in our day.  The paper states:

Learning is not just ‘seat time’ within schools, but extends across multiple contexts, experiences and interactions. It is no longer just an isolated or individual concept, but is inclusive, social, informal, participatory, creative and lifelong. And it is not sufficient to think of  learning simply as consumption, but instead learners are active participants and producers in an interest-driven, lifelong learning process. The concept of a 'learning environment' no longer means just a single classroom or online space, but instead encompasses many spaces in broader, networked, distributed and extensible environments that span time and space. And across these learning environments, learners are offered multiple  pathways to gain competencies and refine skills through open, remixable and transparent tools, resources and processes. In this connected learning ecology, the boundaries are broken and the walls are down — now we just need to help it reach its full potential.
Much of this shift is due to the fact that our world is very different than the one when the current education system was developed and standardized. With the Web and its core principles of openness, universality and transparency, the ways that knowledge is made, shared and valued have been transformed and the opportunities for deeper and relevant learning have been vastly expanded. The open Web has enabled increasing access to information and each other, as well as provided the platform for many  new  ways  to  learn  and  new  skills  to  achieve.  We no longer must rely on the expert authority or professionally-produced artifact to provide us with the information or experience we seek, instead we can find it from peers or make it ourselves online
I believe that the concept of badges is running into a headwind of accountability standards that are counter to its acceptance by academics and businesses, at least in these early years.  Of course it cannot hurt to have champions in the technology industry behind your efforts, but our societal understanding of education and credentialing is not aligned with the concept of badges.  Particularly when assessments proposed to grant the recognition that comes with the badges can be viewed as 'squishy.'  Again quoting from the white paper:
In order for any badge system to accumulate value and for badges to carry or contend with the weight of formal grades or degrees, quality and vetted assessments will be critical. However, the rigor may differ based on the use case, community or intended audience, and badges give us the flexibility to have multiple levels of assessment. Many badges will be associated with distinct pre-defined assessment  exercises and success criteria, whereas others may be more loosely defined and require learner reflection or peer recommendations. The level, or rigor, of the assessment may differ based on the skill. Most hard skills may have fairly standard or rigid rubrics to compare learner work against, whereas social or 21st Century skills will be more fluid and may require more open and social assessments, such as peer reviews or endorsements. 
The intended audience may also determine the assessment level. If badges are simply intended to build community or reward immediate behaviors, as with motivation badges, simple assessments or in some cases, no predefined assessment, may be used. For certification badges meant for audiences such as hiring managers or admission boards, more rigorous assessments may be required to demonstrate critical competencies. Each learner may collect a wide range of badges across many different levels of assessment.
In addition to levels of assessment, badges give us the ability to support open innovation around new or relevant types of assessments, provide more personalized assessments for learners and move away from isolated or irrelevant testing practices. Instead of being forced to take an exam at a pre-determined time, in many cases learners will seek out the assessment on their own, thus encouraging reflection on their learning and competency development.
In other cases, assessment and badge awarding could happen automatically and provide immediate formative feedback, and capitalize on the benefits of 'stealth assessment', which is difficult to achieve in a formal classroom. The badge system also fits well with the increasingly popular portfolio assessment, and in fact creates a distributed portfolio by using the badges as markers or entry points to specific skills and achievements, and each earned badge could then be linked directly to the relevant artifacts in the portfolio.
I wonder how receptive institutions of higher education will be to badges as indications of achieved learning when they have a difficult time coming to grips with effective and fair evaluation of transfer credits from one institution to another.  Accreditation is intended to facilitate cooperative recognition of the quality of the education offered at an institution.  Most accrediting agencies require schools they accredit to accept credits from other accredited schools as a part of maintaining their own accreditation.  The first step to acceptance of these badges will probably require their introduction into accreditation conversations, perhaps through an accreditation of the individual badges, or establishing a process where badges become a required part of the offerings of accredited institutions.  But this concept may require a departure from the more rigorous standards-based and outcome-oriented accreditation standards that are being proposed and implemented now.

In graduate school we are responsible to form a committee of recognized experts in the field of our research who can guide us, evaluate our achievement, and insure that our efforts are sufficiently rigorous and that we make a valued addition to our discipline through our work.  Once they have considered our contribution, and accepted our defense of our efforts, they acknowledge our accomplishment as adequate by issuing a piece of paper that serves as a credential when we represent the adequacy of our achievement to others.  I ponder how badges would serve as a proxy for that credential and process at a graduate level.  As a student I would be in favor of a more open approach to these learning and publishing activities.  As a member of the faculty and as the institution granting the degree, I may be more controlling and traditional.

There is no doubt that there will be (there may already be) progressive institutions of higher ed that are evaluating badge-based assessments and achievement as sufficient for "transfer-credit."  There may even be traditional institutions that have adopted badges as part of their transcript.  But the conservative higher education field and the uncertainty of the real quality of the badges may lead to slow acceptance and adoption.

One final note, in her paper Adrianna Kezar (see Kezar, A. J. (2004). Obtaining Integrity?: Reviewing and Examining the Charter between Higher Education and Society. Review Of Higher Education 27(4), 429-459) talks about how our society is evolving its views of the purposes of higher education.  She laments the changes that are moving higher education from its traditional:
role and contribution to the public good [that] has included educating citizens for democratic engagement, supporting local and regional communities, preserving knowledge and making it available to the community, working in concert with other social institutions such as government or health-care agencies to foster their missions, advancing knowledge through research, developing the arts and humanities, broadening access to ensure a diverse democracy, developing the intellectual talents of students, and creating leaders for various areas of the public sector.
Rather she notes that some critics are concerned that, "higher education is foregoing its role as a social institution and is functioning increasingly as an industry with fluctuating, predominantly economic goals and market-oriented values. Increasingly, the production of workers is the primary or singular goal of higher education."  She goes on to write that one of the concerns about this approach is the "subsum[ing of] the academic functions of the university into its corporate identity."  Kezar quotes Patricia Gumport as writing:
I am concerned that technical, market imperatives run wild, urging colleges and universities to adapt to short term market demands, to redeploy resources, in an effort to reposition themselves with an increasingly competitive context at the expense of long-term goals and commitments.
While there are concerns about this "corporatization" of institutions of higher education, such a definition of higher education as a resource for the marketplace (at least in term of producing professionals for industry) seems to play into the concept of badges.  The marketplace has traditionally relied on the credential represented by a university degree as an indication of achievement and the related major of the graduate as the specific area of skills or discipline in which they have demonstrated sufficient achievement.  However, the emergence of a more need-specific credential may be in line with meeting the exact needs of a company in the marketplace.  There needs to be some deliberation about what these badges mean, who valued issuers are, and the specific achievement that each represents, but it is evident that the separation of the role of the academy in its traditional role for society, and the credentialing for the marketplace, may leave a place for degrees AND badges that may be more aligned with a unique and desired role for each.

1 comment:

  1. Darin, thoughtful as always. I wonder, though, what happens if you decouple badges as credential from badges as "new assessment process." I opined about this some here - In other words, what if all the assessment procedures, etc., remained the same and we thought about badges simply as "a better transcript"? These advances move forward by small steps... if badges can make a "better transcript" play and gain acceptance as a recognition that CAN have value (not necessarily DOES have value), then that opens the door to the democratization of many things.