Monday, August 24, 2009

Interrater Reliability

I was very intrigued by Shannon Hale's recent post about rating books. There are lots of interesting and insightful comments on the post and you should read over them. Here is a part of her post that I have been thinking about since I read it:
"So, I wonder if book evaluation is trumping self-evaluation. I wonder if we get so caught up in gushing or bashing, shining up those stars or taking them away, that the reading experience is weighed too heavily on the side of the book itself and not enough on the reader. After all, reader is more important than book. Reader is the one who changes from reading, not the book. Reader is the one who lives the magic of storytelling."
As a reader, I do feel like I haven't placed enough emphasis on self-evaluation. I should think more about reasons for my reactions to books, especially when I dislike a book. Those feelings and issues sometimes come out when I write a review, but I think those things are more appropriate in my head - not to mention they are much more difficult for me to express. Perhaps if I were a more skilled writer (and had more free time) I could make my reviews more self-evaluative. But, I'm not sure they would be all that helpful for someone looking for a book to read (which is one purpose for this blog).

I'd like to think my reviews are not so much "gushing" or "bashing" but more of an evaluation of what I thought of the book. I think that any review is going to be based mainly on the reviewer's subjective opinion. While I do think it is possible to objectively evaluate some elements of writing, those things are not usually why I want to read a book. I don't seek out books solely for the writer's ability to follow rules and guidelines (though good writing can be a draw, but then again what is good writing? etc etc). I read a book for enjoyment and escapism, knowledge and understanding. Those things can be found in books written with all kinds of styles and even all kinds of ability levels.

My reviews are subjective. I write them to let people know a little about the story, a little about what I liked or didn't like, and what types of possibly objectionable material may be in a book. The overall rating really is just how much I liked or didn't like it. It has no translatable objective purpose. The rating will differ from one person to the next, just as enjoyment will differ.

One of the things I enjoy about reading other book blogs is the huge range of opinions. I can read glowing reviews of a book and then mediocre ones for the same book. This doesn't confuse or aggravate me, it usually piques my interest. If a book has very strong reactions from both sides, I think that is a good indication that I should pick it up and try for myself.

Anyway, do any of you have thoughts on giving ratings or being more self-evaluative and less book-evaluative? Do you look at ratings or skip them over completely?


  1. I do look at ratings, and then read the review. But I don't really take stock in the ratings... they're arbitrary, and I know that.

    I personally don't use ratings, except on Goodreads, and even then, I go back and change them sometimes depending on what I think (or what I've read others think) about the book upon reflection.

    This whole reviewing thing is tricky, though, isn't it?

  2. I always look at the ratings. If nothing else it's like an overview or summary for the review.

    The ones I don't get are the ones where they only give the book 3 stars but the review is all about how much they absolutely loved it! Huh? Then shouldn't it get more stars? I find it very confusing.

  3. Melissa, I think I don't pay much attention to the ratings myself - which is ironic, since I use them heavily. I think it is very tricky stuff - and again, very much subjective.

    Britt, I've probably done that myself. It almost made me want to go back over my reviews and check :)

  4. I look at ratings, but I don't always take them to heart. If I am reading a blog of someone whom I share similar tastes I know I can trust if they liked it then I most likely will.

    I like your rating system, because I do like to know what the swear factor is and what content I can expect. I know a lot of people don't mind that stuff, but to me it taints my view of a book. So I like to know about those things.

  5. Amy, I know I started rating those things for my own benefit, and figured someone else out there might want to to know too! Glad to know someone does :)

    I agree that when I know someone has similar tastes and I trust their judgment, I will pay more attention to their ratings.


Love it when you comment!

Related Posts Plugin for WordPress, Blogger...
Copyright © melissa of One Librarian's Book Reviews 2008-2015