Amid the encroaching “testing mania” in the US, Stanley Madeja (2004) justified the need for authentic, standardized achievement tests in art based on student portfolios. Quantifying art achievement and producing normed test scores, he reasoned, would rely on the same evaluative processes taking place in grading student art and observing their learning and progress “in areas such as visual perception, aesthetic decision-making, critical analysis, visual problem solving, and studio competencies.” (p. 8)
Authenticity is an educational ideal aligning what happens in school to what happens outside of school. Authentic school art experiences parallel the experiences of working artists outside of school. Authentic assessments may improve measurement validity, meaning the measurements correlate more closely to the “real world.” Evaluation is a part of every artist’s growth cycle, whether self-evaluation or as the subject of others’ evaluation, yet in the studies Madeja cites, the criteria artists use to evaluate their own and others artwork differ significantly from the criteria art teachers and their students prioritize.
While portfolio assessments of student artwork have been a longstanding practice in art education, resulting in evaluator narratives and pass/fail decisions, the quantification of student learning in a portfolio, as Madeja suggests, was less common in 2004. Madeja’s confidence in the reliability of judgements made by trained art teachers of all levels is based on studies that show 99% or better agreement between raters about the quality of artworks. He also points to case studies as evidence that quantifying achievement would not stifle creative and artistic thinking in the art class.
Types of portfolio
Madeja describes the following portfolio formats:
Journal portfolios would contain a written log by the teacher and, as much as possible, the student.
Teacher’s portfolios would contain a variety of media and records.
- student work: writings, artwork, audio or video, photos
- teacher assessments on rubrics
- records of student performance [dispositional records]
Controlled task portfolios would contain examples of student work showing growth on teacher-directed tasks over time.
International Baccalaureate (IB) school portfolios would compare student work to exemplars of varying achievement levels in the studio arts. (A current overview of IB art assessments is available online.)
Advanced Placement (AP) portfolios include photos of student work and artist statements about intent and direction of their work. (A current overview of AP portfolios is available online.)
Madeja’s portfolio system
A full analysis of all that could constitute an assessment portfolio, the Madeja Visual Modeling of Information System (MVMIS) coalesces all of these types of portfolio and draws upon examples of creative journalling from art history. MVMIS classifies three streams flowing from the start to end of the portfolio timeframe:
- The acquisition of knowledge stream is the actual collection of data from beginning to end, and may include visual, verbal, and numeric data.
- The analysis and interpretation zone organizes and explains or summarizes knowledge and may rely on diverse visual, verbal and numeric representations.
- The reactions stream, is the place for meta analysis, or reactions to the collected data, and may involve whatever symbol system suits the portfolio creator or audience.
Visual expression throughout
Although portfolios are used for authentic assessment in all fields of education, the significant contribution of Madeja’s system is recognition that visual modes of expression may model or represent ideas in any or all of the three streams. Visuals are the best evidence of the artist’s knowledge, and possibly the best medium for analysis, interpretation, and reaction as well.
Madeja makes a strong argument for quantifying artistic achievement by training individual raters to evaluate portfolios. He supports the argument with evidence that the demand for valid measurements of achievement seem to originate with teachers and artists in and out of school.
Stakeholder accountability aside, evaluation may be meaningful to the developing artist, especially if the artist maintains control over the parameters of evaluation. When an artist selects criteria for evaluation and selects the work subjected to evaluation, that individual secures and maintains artistic agency over the evaluation. School art programs that Take Art Education Seriously empower young artists of all ages to find their distinctive voices.
In the twelve years since Madeja published his MVMIS proposal, the online phenomenon of e-badges has demonstrated democratization and individualization of learning, allowing individuals to acquire credentials along trajectories they find interesting and meaningful. Along the same line it may be possible to allow artists to choose from a menu of evaluative criteria when posting portfolio evidence. Technological developments such as this may add to the value of the proposed digital portfolio system.
Crowdsourcing art assessment
Consider the evaluation of a developing artist’s knowledge and skill were crowdsourced? Crowdsourcing has been defined generally as problem-solving by a large group of individuals using on online platform (Estelles & Gonzalez, 2012). Where evaluators are members of a large crowd assessing the quality of artwork using digital images and artist statements on the Internet, inter-rater reliability may be variable, but computer data systems are well suited to maintaining reliability through ongoing statistical analysis, tracking the reliability of each evaluator and providing them with feedback to improve reliability over time while reporting only the most reliable evaluations to the artist. Evaluators in this situation would be trained on-the-job.
Needs and next steps
Has an individualized, crowdsourced approach to art assessment already been rolled out somewhere online? Digication, Inc., a company that provides an online portfolio system in partnership with the National Art Education Association, does allow for inputing and reporting student assessment data but does not advertise the system’s potential for individualized or crowdsourced assessment. There may be a need to find out to what extent other existing online portfolio programs such as IB, AP, Scholastic Art Awards, or Artsonia provide for art assessment.