1 2 3 4 5 STARS BIAFF 2013

[Mike Shaw]

Thanks Willy - good to know these films are still being seen - and enjoyed! I think I mentioned earlier - I believe audiences are the best gauge of a film's 'worth'. With Flat Out, which I edited (and was one of the three camera people), there are over a dozen continuity and other 'shooting' errors in its 7 minute length... I have mentioned this on several occasions to different audiences before showing the film, asking them to see how many they can spot. Sometimes, if I'm lucky, someone will spot ONE. Why don't they get noticed? Because people follow the story - not the minutiae.

I can understand the desire to 'strive for perfection' in a hobby - we naturally all want to do our best when creating something. And it is unquestionably important to pay respect to the technical aspects of movie-making. But it is the creativity behind the movie that people tend to remember rather than its technical aspects. Interestingly, I recently saw a 'joke' film that I had seen previously made by someone else some time back. Same joke - but chalk and cheese in their execution and creation. One was tackled with flair, the other plodded through the story with limited camera angles and went on after the punch line (with a series of unnecessary mock reaction facial expressions).

I think the ratio "... 1.5 for technical quality, 3.5 for the appropriate use of technique and up to 6 for message/story" - (the way the story is told - the creativity?) - mentioned by Geoff is probably a reasonable one to consider when judging movies. I believe general audiences tend not to see the flaws in the technical aspects - only those in the story line. Too slow, too tedious, too ponderous and too plodding and they'll happily nod off.

(Guess that's why I like making short films - don't give 'em time to snore! )

[TimStannard]

I think the ratio "... 1.5 for technical quality, 3.5 for the appropriate use of technique and up to 6 for message/story" - (the way the story is told - the creativity?) - mentioned by Geoff is probably a reasonable one to consider when judging movies.

I don't believe you can divorce "the appropriate use of technique" from "the way the story is told - the creativity". The example you cite of two versions of the same story provides a perfect example of this (we've all seen this). Limited camera angles, poor sound, bad lighting, inconsistent exposure, poor acting, etc could all be considered "application of technique" as much as "creativity".
For this reason I don't believe one can or should attempt to give "weighting" to different technical/creative aspects of judging a film. And this, of course, is one reason why it is so difficult to get two panels of judges to agree.

[Mike Shaw]

Hmmm. Interesting. I see lighting, exposure and so on as being technique and technical ability rather than creativity - but I'd definitely put camera angles down as creativity, particularly when they 'break away from the norm. Some say, for example, "shoot at eye height". But the interesting shots, I think, are not always at eye height.

There's no doubt that the two - technique and creativity - are intertwined. To me it's what sets Bob Lorimer's films, for example, a healthy cut above the rest: the plot is almost secondary to the technique and creativity in his movies: I've yet to see one that has failed to entertain throughout, a joy to watch. A great blend of technique and creativity.

Our club (OVFM) has 'score sheets' for use when the audience is judging films. There are about 5 different items - Sound, titles, editing and so on, which are scored up to 10 points each: one is supposed to rate/score those irrespective of whether you like the film or not. Then there is a General Appeal category which can have up to 25 points - which definitely does reflect how you feel personally about the film. It means a technical competent film with absolutely no appeal whatsoever (to one individual) could score as high as 50 points, while another with oodles of appeal but a pretty low technical ability scores could score almost, if not as high. Which is the better film ... the competent one, or the appealing one. I guess they have developed this method of scoring as being the best after years of experiment (been going for over 50 years!).

No, I'm sure you're right. Divorcing technical skill from creativity isn't ideal - but, with something as subjective as a movie, where does one start trying to tie it down to a 'rating'.

[Dave Watterson]

Just a thought:

I see competitions and ratings in a club context as part of the learning/educational process. It might, therefore, be useful to signal technical issues so that members can improve their technique.

But comments should never just be on technique.

It always helpful to respond to a film just as most of us do to a television programme: that was rubbish / that was impressive/ I was surprised when ... / I cried. That is giving a human response to the information, mood or story in the film.

Ideally one wants both at club level.

At National / International level the only time I worry about technique is if something distracts me from being totally engaged in the film. The notion of looking out specially for technical "failings" is a waste of energy and attention.

[TimStannard]

We (Staines) used to have a similar, though not identical, system to OVFM. Marks out of 10 each for visuals, sound, editing and out of 20 for "general".
A few years later (funnily enough when I joined) we got rid of that, for three reasons:
1. Several members didn't feel knowledgeable enough to score what we might call the technical merits (and actually disliked being obliged to do so)
2. Several film makers thought there were many members who weren't knowlegeable enough to score the technical merits.
3. Some people thought technical merit should count for, say, 70% of the score whereas others thought it counted for only, say, 10%.

The result was that for many scorers, the values tended to be the same across the board (eg 8/10 each for visuals, sound, editing and 16/20 for general)

So, we simply did away with it, allowing people to score each film out of 50 based on their own values.

We still have columns for people to give marks out of 10 for the technical merit categories but completeing these is entirely optional and they do not count towards the film's placing in the competition. In practice only about a third of the members present fill in these categories. They are totalled and fed back to the makers as this is considered useful feedback, coming as it does only from members who chose to feed it back.

[Dave Watterson]

This touches on a general problem with all marking/scoring systems: hardly anyone uses the full range of marks available.

There are all sorts of reasons for this, but it reduces the value of such systems and sometimes makes a mockery of them. At BIAFF I was shown a mark sheet with more than 70 items on it. For every film, judges were expected to give a mark out of ten for each item. It is a standing joke in that country that "all marks are 7".

It would be cruel - and untrue - to say that any particular film was worth no marks, or that any particular film was worth full marks. But if you were to rank all the films in a small competition one would be the worst and another the best. If you privately allocate 0 and 100 to those films, you can probably rate all the others along the scale between those two points in a way which shows, perhaps, that the top two films are streets ahead of the third one, but there are, say, two films very close to the worst one.

On a more practical note: what is more helpful to a film maker: a score or some comments, however poorly expressed?

Perhaps we need to encourage / train club members to give brief spoken comments on films rather than worry about ways of finding a number or rating ...

[TimStannard]

This touches on a general problem with all marking/scoring systems: hardly anyone uses the full range of marks available.

Too true. Many allocate all films somewhere between 40 and 45.
I tend to give a low around 20, but why? Some strange idea that it is 40% better than a flim which is just a blank screen with no sound?
We have others who come close to your idea of 0-50 (in the case of Staines) but then if we adopted that strategy (which actually makes perfect sense) we'd have one poor soul who would always score zero. At least with the present system he can see his score increasing, even if he alsways comes last.

Which brings us on to:

On a more practical note: what is more helpful to a film maker: a score or some comments, however poorly expressed?

Coming last doesn't help anyone. Comments are always more useful. That is why we always have both spoken and written ones (which are put up on our forum for all to see). What I have learned about this pastime/hobby/obsession of ours, I have learned mainly through reading and listening to other people's comments about my own films, but moreso about other people's films.

However, if we are indeed to have a competition, we need a way of determining a winner and you cannot achieve that without some means of scoring/rating.

[Mike Shaw]

I was told by a very experienced judge, on my first county level 'judging session', that when the objective is 'picking a winner' (i.e., comparative judging), always mark the first film in whatever categories you like to break your own assessment method into, at 50%. Thereafter, other films are better, or worse, by a specific 'percentage rating', and at the end, hopefully you have just one with an overall highest score.

BIAFF judging though is not comparative in the first 'round'. It is in effect, 'marking' the skills and creativity of the film/maker in order to come up with a 'star rating'. Then, only those rated in the 5 (and 4) star ratings are moved on to the final judges to ascertain what, if any, 'prize' awards they may win - and whether they go on to become a 'Diamond' winner.

Selecting the star rating for a movie is very much a 'tick box' situation - you can read about what the stars mean on the IAC website, which should give a guide as to what the tick boxes are.

To me, the star rating is a guide of a film's technical aspects. An audience's reaction is the guide to it's entertainment value.

I think it is sad we have to be in a competitive arena when it comes to our hobby - I make films to be seen, not to win trophies, but appreciate a selection process is absolutely inevitable - if only so that programme selectors can put together a 'balanced show'.

And as long as there is a selection process which relies on opinion rather than 'whose car passed the winning post first', there will always be disagreement with the choices of winner and so on.

In this thread though, the disagreement is also with the star rating, which is even sadder. But then again, even with 'tick boxes' I'm afraid subjective opinions will shine through.

I and two others have recently finished a film we've been working on for over 7 years ... at BIAFF (next year perhaps), I hope to get no lower than 3 stars for it. But the good news (for us) is that organisations - various Societies and Guilds - are asking us to show it to their members, and the excellent audience reaction we receive makes it a well-worth while exercise.

Now, what better reward can one have than that?

Which, I am afraid, is all a very long-winded way of saying - comments will always be better than scores....

[col lamb]

Interesting.

The fact that some of you guys are using score sheets, when I suggested this a few years ago as a way forward to improve standards and feedback it was shot down by all.

First and foremost a movie must entertain, be educational, be informative, be a pleasure to watch, be stimulating........etc, etc

As one of the premier competitions in the world our BIAFF should expect high standards in any movie getting a rating above average, to be awarded 4* and above a movie should provide the creative requirements/enjoyment mentioned above however it also should be technically to high standards of editing, timing, composition, cinematography, sound, colour balance, etc and if there is more than the odd technical error then the movie should not achieve the higher status award.

Interesting Mike should mention audience reaction we held an IAC NW competition a few weeks ago where the movies were judged prior by an IAC judge who lives outside the NW the winning movie was one that got 3* at BIAFF and on the night 75% of the audience voted for a completely different movie, incidentally I agreed with the audience vote for the best movie.

Its not the star rating that I have problems with its the allocation of inappropriate stars to a particular movie and that may well be too few as to many.

[TimStannard]

I think it is sad we have to be in a competitive arena when it comes to our hobby

But we don't. No-one is obliged to enter competitions. We have many member who show films but don't enter them into competitions. They receive the all important comments, but none of the grief associated with competition.

Similarly, many of us enter BIAFF not in order to win anything as such, but (a) to receive comments from judges and (b) to see whether, in the eyes of the judges, we have made improvements compared to previous years. Friction arises when we see a film which we clearly think is good scoring badly compared with a film we think is bad. Perhaps we'd all be happy if we were only told our own star ratings. Maybe not.

And don't forget, no matter how much Michael Slowe may hate the format, there are plenty of forums on the internet where one can submit films for others to criticise (and not all are hostile - certainly not the one I moderate, or the fledgling section on this forum). Also there are the Red Carpet Screenings down in Basingstoke (and presumably others around the country) where films are shown and discussed in a non-competitive environment. Having said that, this is still to some extent competitive. Someone, somewhere has to decide which of te films submitted wil be screened.

Which, I am afraid, is all a very long-winded way of saying - comments will always be better than scores....

I doubt there are many who'd disagree with you there, Mike.

[col lamb]

In the latest FVM there is a breakdown of the 257 BIAFF 2013 entrants and it is interesting that 53 (c20%) of the entries are from overseas and of these 53 entrants 10 were awarded a Diamond and another 17 gaining 4* and only 9 awarded 2* or 1*.

So more than 50% of the overseas entries gained a high rating.

Those who are not happy with your award now know what to do

Wooden spoon to EARIAC, for 4 entries this year and 6 last, come on you guys in the East get making movies

Also a different type of star should be awarded.

Extra "effort stars" to those in Scotland and the North West as they are the only two regions where the number of BIAFF entries this year is greater than last year.

I have also had a detailed look through the BIAFF movie entrants and also if any the judges of my two movies were active, I'll sort of protect them by using A, B, C etc instead of their names: -

Judge
A - 1 movie 3*
B - 1 movie 2*
C - 0 movies, shame on you
D - 2 movies 2* each
E - 1 movie 4*
F - 1 movie 3*

Make of that analysis what you want !!!

Finally well done to David Newman and his team for handling so many entries and to CEMRIAC for the great presentation of the movies

[Mike Shaw]

I think it is sad we have to be in a competitive arena when it comes to our hobby


But we don't. No-one is obliged to enter competitions.

Oh, but you are if you want your film shown at festivals - with their larger-than-the-local-club audiences.

[Dave Watterson]

Col, you must type some of this with your tongue in your cheek!

We in the UK are encouraged to send in entries even if realistically we think they may only get one or two stars, it is a way of measuring our progress as film makers. People taking the time and trouble to enter from other countries are usually those who have had success already in their own lands, so we should expect their general standard to be high.

EARIAC has 6 clubs and an archive! Compare with SOCO which has 38 clubs. I take the point about a reduced number of entries, but that will vary from year to year ... some years people feel inspired more than in others. (Discuss!)

Dave >

[col lamb]

Dave

The tongue is frequently in my cheek but it is not with the comments below.

At BIAFF I watched the following overseas entries: -

SOS Plumbers
Marc Remembers Sarah
Lipstick
Four Way Stop

These were awarded 4*, to me three are 3* and one is a 1* (and I am being kind on the 1*)

Since BIAFF I have watched online: -

Presents from the Past
Mindbreak

Both of which were excellent and deserved their alocade

Over the years I have seen very many overseas movies and the vast majority just simply do not deserve the rating they received. One of this years movies had such bad lipsync it was embarrassing to watch, it was about 3 seconds out. If it was supposed to be funny then it went right over my head.

Point taken about the number of clubs in a region

One thing I will always shout long and hard about is that judges should be active movie makers, what I did not expect when I looked at what the judges of my two movies had received from their own BIAFF entries was the ratings. Six judges, six movies, one 4*, two 3* and three 2* this is by anybody's standard nothing other than surprizing.

I have just looked through those who were judges and of the 15 judges the highest any of them received was a single 4* for Tom Hardwick's entry so the questions have to be asked.

[TimStannard]

I think it is sad we have to be in a competitive arena when it comes to our hobby


But we don't. No-one is obliged to enter competitions.

Oh, but you are if you want your film shown at festivals - with their larger-than-the-local-club audiences.

I think the NTR festival has this right. All films receive judges reports and winners - the top three (plus occasional commended) and special category (eg editing) - are awarded anything. No scores, ratings, or whatever are given. Last year all but three entries were shown - and I can make a reasonable assumption that the three films which were not shown were not because they were the "weakest", but for other reasons (like the makers already had several films being screened).