{"id":9139,"date":"2015-09-30T15:41:29","date_gmt":"2015-09-30T15:41:29","guid":{"rendered":"http:\/\/blog.studyabroad101.com\/?p=9139"},"modified":"2019-12-04T07:38:57","modified_gmt":"2019-12-04T12:38:57","slug":"why-imperfection-in-a-study-abroad-evaluation-is-ideal","status":"publish","type":"post","link":"https:\/\/www.studyabroad101.com\/blog\/2015\/09\/why-imperfection-in-a-study-abroad-evaluation-is-ideal\/","title":{"rendered":"Why Imperfection in a Study Abroad Evaluation is Ideal"},"content":{"rendered":"

Abroad101 has many many reviews from students. The study abroad reviews are meant for study abroad students, their schools and future students to analyze the program they went on but also to help the student reflect and understand what they learned from their time abroad. Future students and parents find reviews very helpful when trying to decide which program to pick for an upcoming study abroad trip. With this in mind\u00a0the quality and variety of the review comments and ratings becomes very important.<\/p>\n

Our guest blogger, Missy Gluckmann wrote an article about this concept. It is reposted for you here.<\/p>\n

Education Abroad Evaluations: Why Imperfection is Ideal<\/h1>\n

\"Truman<\/a>The culture of higher education is to want to measure.\u00a0 We want to know not only how people have been changed by education abroad experiences, but how they FEEL about them too.\u00a0 Assessment has grown exponentially in our field and we sometimes get stuck on the metrics over the human experience \u2013 at least that is my opinion. But this post is about evaluations, so let\u2019s talk about them.<\/p>\n

I recently had a conversation with Mark Shay, CEO of Abroad101.com<\/a>, about evaluations and what they really mean in our field these days.\u00a0 Let me back up though and share some history; Mark and I have worked in international education since the 1990s. While we did not work \u201cdirectly\u201d together on a specific project back then, I clearly remember the early days of his educational entrepreneur ventures at studyabroad.com.\u00a0 Back in the day (wait, am I really old enough to say that \u2013 yup, apparently I am!), this was \u201cthe tool\u201d for finding education abroad program information. Today, he is leading Abroad101.com which is described as follows:<\/p>\n

\u201c With more than 20,000 reviews of 3,800 programs, Abroad101\u00a0is the leading student review site for study abroad programs. It\u00a0provides a free service for universities and students to rate, review and rank\u00a0student experiences in study abroad programs.\u201d<\/i><\/p>\n

It is a timely business idea, as education abroad offices are tasked with creating evaluations, measuring \u201csuccess\u201d and increasingly looking to student satisfaction in this game of extreme competition that has evolved in higher education and sadly, has trickled down to the education abroad world.\u00a0 Ironically, small education offices often don\u2019t have much time to think about evaluations, and those that do often don\u2019t have much time to make meaning of the data or simply use it to pull \u201csounds bites\u201d for marketing.<\/p>\n

The \u201cbigger players\u201d tend to evaluate, sometimes not allowing students to receive their credits until a mandatory evaluation is completed.\u00a0 I find this to be a challenge, as I believe that time to properly reflect on an experience abroad is more important than keeping to a timetable. Withholding grades, in my opinion, borders on unethical.<\/p>\n

Mark and I had a chat about the direction of his company\u2019s website now that he is \u201cofficially\u201d on board as CEO.\u00a0 We got into an important and sticky topic \u2013 why do so many programs come up so very high on ratings?\u00a0 Why are there so many \u201cperfect\u201d ratings in education abroad reviews?<\/p>\n

We know that education abroad experiences, like many things in life, cannot reach perfection or close to it. The lack of funding to train faculty to encourage participants to do what Dr. Anthony Ogden describes as \u201cgetting students off the veranda\u201d, is evidenced by the countless \u201cisland\u201d experiences that we witness on faculty led academic courses abroad. We see comments by students about \u201chow transformed\u201d they are, yet how does a perfect five star rating illustrate how a program\u2019s design has positively impacted that young mind and the abroad experience? This is where the rating (\u201cfive out of five stars\u201d) of a program can diverge from the actual review of the details of a program \u2013 and the \u201cdevil is in the details\u201d!<\/p>\n

What is \u201cperfection\u201d in education abroad?\u00a0 Can we truly be \u201cperfect\u201d at anything in life?\u00a0 Does a \u201cperfect\u201d score mean we are doing all that we can to create an optimal learning experience for young minds? Or does it mean we are not setting a high enough bar or that students don\u2019t have high expectations \u2013 or even enough life experience to have realistic expectations across cultures? Does a lack of maturity prevent students from providing deep and thoughtful critical analysis?<\/p>\n

More precarious is that there may be pressure from program providers and administrators to keep the scores \u2018high\u2019 \u2013 as some education abroad departments are embracing the competitive nature of higher education admissions\u2019 philosophy, transferring the demand to perform to those going through re-entry by offering an incentive to them to complete a positive evaluation for a prize of some sort (bookstore bucks, access to academic records, etc).<\/p>\n

Another challenge is that students haven\u2019t been given much guidance in HOW to actually review a program so that information can be gleaned that will actually help improve the program design. I recentlywrote a guest blog post<\/a> for abroad101.com about this subject.<\/p>\n

During our chat, Mark and I quickly agreed that giving (or coaching your students to give) a \u201cperfect\u201d score on a program is not useful to those who are researching future participation in an experience abroad.\u00a0 Giving someone a score of five on a site like abroad101.com is not \"Abroad101_logo_minus_com_2\"<\/a>like jumping on a site like yelp.com and letting someone know that you thought a particular meal in a new restaurant was \u201cspot on\u201d and to your liking on a given day.\u00a0 Education abroad is so much more complex than one random stop for brunch.\u00a0 It requires that we are critical and thoughtful in our feedback and that we specifically disclose the imperfections to provide accountability and necessary education for others who are considering these programs.<\/p>\n

Let\u2019s consider an example: A student in Rome completed an evaluation and ranked it \u201cfive stars\u201d (perfect score), yet she indicated that she would not return abroad with the same program.\u00a0 When reading her evaluation further, she illustrated concerns about program administration, yet still rated the program with a perfect score.<\/p>\n

Confused? So am I.<\/p>\n

My guess is that someone in her study abroad office failed to take her to task on the incongruous feedback. It would have been so much more helpful is she had actually taken her comments on the program administration and carefully weighed them against the scale, providing a more realistic overall rating score, perhaps a three out of five.\u00a0 This would then prompt future readers to consider the program weaknesses, which really are opportunities to address issues and improve processes and outcomes.\u00a0 These types of scores are the ones that create REAL dialogue about program design and delivery of service to an academic sojourner.\u00a0 They also open up the door for discussion about partnership between universities and third party providers (who I prefer to call third party partners \u2013 as that is what the relationship should be based on \u2013 partnership\u2026but I digress).<\/p>\n

I have seen a similar example for a third party provider in Tanzania. The student rates the program a five yet consistently ranks it lower in all subcategories, including a \u201cone star\u201d on the academic experience with this commentary:<\/p>\n

\u201c<\/i>The academics were overall pretty terrible, whenever the professors did actually show up they usually did not even teach material that was relevant.\u201d Regarding housing, she says \u201cThe dorms are pretty terrible and my roommate didn\u2019t even live here so I was by myself. The safety on campus is not great and you can\u2019t walk alone after dark. It\u2019s about a 15 minute walk to class and a 30 minute bus ride to nightlife. There is also no access to kitchens\u2026.and that was a big problem.\u201d<\/i><\/p>\n

It begs us to ask how THAT evaluation was overlooked by the home university and permitted to land on the site in such condition.<\/p>\n

A score of five indicates to others that a program is stellar. Perfect. Wouldn\u2019t change a thing. Except that in some cases, there IS much that needs to change.\u00a0 Sadly, we miss the opportunity to get to the fine detail about what those components in need of work are when we see imperfection as failure.<\/p>\n

Had the students rated the programs with more accuracy (e.g. \u201cnot perfect\u201d), it would encourage not only administrators but prospective students to more carefully consider the actual \u201creview.\u201d A score in the \u201chigh fours\u201d is much more revealing to someone seeking information on a program.\u00a0 It offers a positive endorsement of the overall program yet provides specific details of what to expect, what could change to improve the experience and what to consider when making the important decision of who to go abroad with. Its value is priceless when compared to the ubiquitous \u201cfive\u201d, as it actually provides insight into the little nuances that are so important (e.g. what to pack that was not mentioned in pre-departure materials\/orientation) and the bigger ticket items (e.g. feedback on where budget improvements can be adjusted by a provider to allow students to more consistently engage in activities throughout a period abroad, vs. blowing all of their money during the first two weeks).<\/p>\n

I realize that someone who rates a five may be someone who feels they had a positive, \u201clife changing\u201d overall experience abroad and wants to communicate that to the masses, but as Katy Rosenbaum from the Melibee team stated so eloquently, \u201cI think it\u2019s safe to say that a great time does not necessarily equal a great program.\u201d\u00a0 Frankly, we all know of less than immersive education abroad programs that are highly rated.<\/p>\n

Perhaps this quote by Iain Thomas sums it up best when it comes to education abroad evals:<\/p>\n

\u201cBut life isn\u2019t something that should be edited. Life shouldn\u2019t be cut. The only way you\u2019ll ever discover what it truly means to be alive and human is by sharing the full experience of what it means to be human and each blemish and freckle that comes with it.\u201d<\/i><\/p>\n

Give me imperfection over a perfect score any day.\u00a0 It is those blemishes and freckles that will inform and make the evolving world of free evaluation services in education abroad a truly meaningful tool for all of us.<\/p>\n

 <\/p>\n

\"Missyheadshot2.jpeg\"<\/a>About the Author:<\/em><\/strong>\u00a0Missy Gluckmann is the founder of Melibee Global.<\/em>\u00a0<\/em>You can\u00a0learn more about why she built Melibee Global and her background here.<\/a>\u00a0\u00a0<\/em><\/p>\n

 <\/p>\n

 <\/p>\n

 <\/p>\n","protected":false},"excerpt":{"rendered":"

Abroad101 has many many reviews from students. The study abroad reviews are meant for study abroad students, their schools and future students to analyze the program they went on but also to help the student reflect and understand what they learned from their time abroad. Future students and parents find reviews very helpful when trying […]<\/p>\n","protected":false},"author":21,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":"","_ppma_block_editor_authors":""},"categories":[1],"tags":[],"ppma_author":[684],"class_list":["post-9139","post","type-post","status-publish","format-standard","hentry","category-uncategorized","author-abroad101"],"aioseo_notices":[],"aioseo_head":"\n\t\t\n\t\n\t\n\t\n\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t\n\t\t