# SOTM - voting approach considerations



## mattwalt (Jan 5, 2017)

*SOTM Votes*​
*This is the Question...*

To Vote631.58%Or Not to vote1368.42%


----------



## mattwalt (Jan 5, 2017)

I have received some criticism for choices I have made with regards to the winners of certain of the SOTM's - and there has been a few instances where members have requested to return to a voting system.

I have a weighting process I follow - aesthetics / perceived shoot-ability / materials used / approach / technology... As a result the most blinging frame isn't always the one that makes the cut. So I'm sure in some cases people may be left scratching their heads in bewilderment  Though as its currently up to me I stand by those decisions.

I am pretty loath to offer a votable platform as its been a contentious issue and was open to some severe abuse in the past. When I restarted SOTM I realised almost instantly by removing this 'function' the results would be more fair. Ironic - but thats the way it is.

But in fairness I will offer it up to the members whether they would absolutely prefer a vote option for further SOTM. One member suggested looking at a platform which would allow non-members the choice to vote as well... Which I can look into.


----------



## IOAN (May 25, 2020)

It's not clear to me what to vote for !?


----------



## mattwalt (Jan 5, 2017)

Ah - sorry 

In the original SOTM the frames would be collected - and members could vote for their favourite one. The most votes would win.

Though people would have their friends vote on their behalf - sometimes bullying them. Also there would be 'spats' on approach - natty's were generally considered as degenerate vs anything cored etc.

Let's say it got messy - messy enough for almost half of the forum members to either have left the forum or were outright banned.

When I first took over SOTM I did a vote to win type approach - though it was evident by the 4th month that the system was being abused. So I stopped it by the 6th month. And as such when I relaunched it after much consideration I set up certain parameters to keep the whole thing fair and be possible to have longevity. Voting was one of the first things I dropped and assumed the burden of a personal judgement.

So choose carefully


----------



## Pebble Shooter (Mar 29, 2014)

When one person evaluates and decides, well, you get China....err, not good.

Voting extended to the wider forum readership, most of whom are non-members, would solve the inherent bias that comes from member votes only, where people generally know and invariably support each other.

Judging by the overall numbers (thousands) of views of the monthly competitions, the number of non-member votes could constitute a large enough sample to generate a truly objective assessment of the work submitted by competition participants.

A non-member vote may also encourage more non-members to become active members, thus creating a stronger presence of this forum on the web.

Why not give it a shot?


----------



## mattwalt (Jan 5, 2017)

Just know - it could be a SOTM killer


----------



## Island made (Aug 14, 2019)

I'm fine with it either way Matt, I'm an easy going fella and I really enjoy what your doing with this competition. Altho I might not agree with every winner you choose I agree with how you choose a winner. And if it were me judging I can guarantee that there would be a lot of people disagreeing with me

If there's one person to judge a competition it's you. The problem I see when it comes to judging is that it becomes a popularity contest. I for one have a lot of good friends on here and I can say that it would be hard to vote on a frame that they didn't do even if it were the "nicer" frame.

The last ssotm burnt out because of a voting problem...were a year into this one and it's still going strong IMO. No matter what platform you choose there's gonna be hair bent folks....just the way it is and we'll always be.


----------



## mattwalt (Jan 5, 2017)

It was suggested placing the voting platform somewhere publicly accessible - so that it's not just members that have access to the voting. Which could help maybe.

Think the one issue is that actual member wise we are a fairly limited bunch - so making choices very quickly can sway results.

TBH - some months literally result in ulcers. Weighing up criteria and approaches...

Also ts worth mentioning that sometimes the best looking frame can have some less than desirable fit and finish - its easily missed if not scrutinised carefully.


----------



## Pebble Shooter (Mar 29, 2014)

Mattwalt: "just know - it could be a SOTM killer."

I don't think so, because it's one thing having long-term forum (fictive!) members Joe.Jack, and Jane voting, or even bullying (!!!) for each other to get the prize, and another ballgame altogether when you have a fairly vast number of voters consisting of forum members *and non-forum members*, because the likelihood of all non-members actually knowing each other to the extent that they would "lobby" for a forum friend is somewhat lower.

It's a bit like the difference between living in a small village where everyone knows each other, and living in a huge city where most people are largely anonymous. The problem with the village is that the chief and his cronies tend to get what they want, regardless of the "underlings".

Another way to put it is the classic "old boys network" where one hand rubs the other, as opposed to the wider, more amorphous public when it comes to voting on crucial issues in companies or in politics. According to the way you describe your past SOTM experiences, it seems as though the former must have ruled galore. :hmm:

As they say, nothing ventured is nothing gained: maybe it's worth trying another approach for a while. If it fails, simply revert to plan A (like now). However, the way forward will of course ultimately depend on whether the current forum members are happy with the present SOTM format or not.

I'm curious to see what the others think about this issue.


----------



## mattwalt (Jan 5, 2017)

PS - I'm happy to give it a go - just wanted to get a gauge of the Tribe 

Some of the older members who survived the original may understand why I did things a certain way.


----------



## Crazy Canuck (Dec 20, 2018)

As someone who just participated in their first month, I really enjoyed the process. I appreciate all the work Matt takes on to make it happen each month. With the issues that plagued the event in the past, the current way seems refreshing and fun.

From my perspective, as it's not a commercial venture, I'm not in it to win it. It's more of pushing myself, and enjoying what others create within the criteria.

It almost feels more casual this way, and as a result people are more free to enjoy the processes rather than take it too seriously.

If it goes the way of voting, I'd still be fine with it, but hope it wouldn't kill the current community spirit that it has now.


----------



## mattwalt (Jan 5, 2017)

TBF - Nathan (Simple Shot) and Bill (Pocket Predator) and think Game Keeper John and many other 'big' names used to be part of the original - and quite a few other well known custom makers. Some of them definitely used the platform as advertising for their work. So there is the potential to have it be come a financial bonus. Which was one of the reasons members were being aggressive in collecting badges.

I like the fact that currently is pretty light hearted - and thats mainly due to the members who participate. Though bummed Mo's entries have started to be pretty darn slick these days...


----------



## Ordo (Feb 11, 2018)

I like monarchies and soft dictatorships, so I voted not to vote (which is something paradoxical).


----------



## Crazy Canuck (Dec 20, 2018)

Another option is to continue to include guest or volunteer judges like with the bark on natty month. A panel brings different opinions and viewpoints.


----------



## devils son in law (Sep 2, 2014)

I say put it to a vote, hopefully it will keep you from any criticism. I hope none of it was personal, there's too many great people here to be tainted by a few.


----------



## mattwalt (Jan 5, 2017)

LOL - I have asked a few people to create a panel 

People seem to run for the hills. If anyone wants yto PM me - I'll be keen


----------



## SJAaz (Apr 30, 2019)

Hey Matt...

In my opinion you started this. Without you, there would be no contest. I say you do what you want, if folks don't want to be involved, don't enter a sling. No vote for me.


----------



## mattwalt (Jan 5, 2017)

Honestly I'd think a panel would be the way to go. Entries could be discussed and critiqued - a bit more Forged in Fire. Think it would be the most beneficial and fair. Though the judges would need to be builders of some skill and would not be able to enter their own work.

I do understand the hesitancy of interest - it opens people up to a degree of criticism and abuse.


----------



## Quercusuber (Nov 10, 2011)

* "natty's were generally considered as degenerate vs anything cored etc."*

...I knew I was a degenerate pig!! :rolling: :rolling:

Matt, this is all fun and games. But you've shown you're a correct and stand-up guy in posting this question to the ballots.

There's only one thing that unite us all in this forum: our passion for slingshots, whether for shooting them or building them.

Something beyond that should always be disregarded by the members as belonging to a different set of values.

Best regards!!

Q


----------



## Harry Knuckles (Sep 28, 2020)

Maybe a voting/scoring with rubric?

aesthetics (Circle Your Score) - 1 2 3 4 5 - Provide a brief explanation of your score _________________________

perceived shoot-ability (Circle Your Score) - 1 2 3 4 5 Provide a brief explanation of your score _________________________

materials used (Circle Your Score) - 1 2 3 4 5 - Provide a brief explanation of your score _________________________

approach (Circle Your Score) - 1 2 3 4 5 - Provide a brief explanation of your score _________________________

technology (Circle Your Score) - 1 2 3 4 5 - Provide a brief explanation of your score _________________________

Then average out the final scores and total.

Just a thought.


----------



## mattwalt (Jan 5, 2017)

Also Fit and Finish... Thats been the downfall of a lot of the most likely winners.

And answering the brief...

But thats sort of how the scoring works - though in some cases certain criteria weigh slightly more than others. Almost like bonus points 

I'd love to giver everyone a long explanation of what the scoring is/was and why - but TBH I'm quite a busy boy most of the time - and going the full long haul explanation isn't feasible. Though I have often offered the ability for entrants to PM me to discuss - I'm usually happy to explain why something they thought was a frontrunner was chosen over something they felt was far superior.


----------



## mattwalt (Jan 5, 2017)

OK - so one member has hit me up and offered their support as part of a panel. You know who you are 

If one or possibly max 2 more would be keen that would be awesome.

What I may do is keep the panel secret... Though anyone on the panel would not be eligible for consideration for a given month.


----------



## flipgun (Nov 14, 2012)

Stand your ground! It's your bat, your ball and it is in your yard. A voting system eventually leans into fans of,,,?... carrying the game.

Better the decision of an individual than the disputes of the hoi polloi however good-natured.


----------



## Reed Lukens (Aug 16, 2020)

Harry Knuckles said:


> Maybe a voting/scoring with rubric?
> 
> aesthetics (Circle Your Score) - 1 2 3 4 5 - Provide a brief explanation of your score _________________________
> 
> ...


There's 2 sides that I can see. If you did an open voting system open for anyone like HK shows above, then it would be a no brainer on who wins... but will people take the time? Not having to vote makes it easy on us and puts it all on you Matt. I say, it's up to you. If you're getting to much feedback that isn't helping you smile and stay happy, then ship it out to the members. I'm not keen on opening it up to the public myself, but it's a tough call... then if we all vote 5's for everyone, whose going to break the tie? Matt... that's who


----------



## mattwalt (Jan 5, 2017)

I'm just testing the water at the moment. I'm quite content to let it keep running the way it is - I'm not concerned about the fallout all that much.

As mentioned someone has offered to help moderate this month - which is total awesomeness. Thats already a win IMO. Accountable deniability...

The other thing worth pointing out with a poll/vote is that it makes sense to only choose one image - which can be misleading. Also it could become a burden time wise simply setting up. But it's doable. Not that this has much bearing on my general concerns about a voting system.


----------



## MOJAVE MO (Apr 11, 2018)

Set it up like judging a boxing match. 3 judges. Points per round both for and against the frame entry per judge. Judge anonymously. Add the points. Or maybe a cage fight to decide the winner!?

Sent from my iPhone using Tapatalk


----------



## Pebble Shooter (Mar 29, 2014)

Something I repeatedly come across in online media these days are regular polls to ascertain what people (the readership) think about approach A or B to a specific current controversial issue. Although the way a question is formulated can indeed influence the public readership, it seems to me that the judgement of a higher number of people (who do not know each other) is always preferable to a very limited number of judges, where the risk of collusion is inherent.

A clearly understandable range of criteria with click-boxes ranging from, say, 1 to 10 points, next to a close-up photograph for each slingshot being submitted to a monthly SOTM contest, where voting is also accessible to non-members, would most certainly yield an objective opinion to decide who wins the badge. I don't see where the problem is with such an approach, not to mention that it would alleviate the workload for the moderator launching the contests: some well designed SQL would automatically calculate the final results every month.

There is no reason why such a voting system would adversely affect the SOTM "fun factor", because quite on the contrary, it is more satisfying to have the approval of a wide range of slingshot enthusiasts that minimizes the "friend of a friend" impact. Either way, in my opinion a one judge system simply is not objective: just because I really like vanilla ice cream does not mean that everyone else does as well.

I would give the expanded voting system a try, just to see what happens. No pain, no gain: We might all be positively surprised at the outcome. B)

Thoughts, anyone?


----------



## Tremoside (Jul 17, 2013)

Hi Everyone,

I think an Editor's Choice and Community Award are both good options. I have seen both hacked, abused and messed up over my forum experience since 2003 and couple thousands of posts overall on other places too. I think the Editor's Choice is usually standing better over time.

So I support the picking of a winner instead of voting, but voting is fun and meaningful for the community. Both of them have the value.

Also many forums using a plug system to display winners on frontpage. That is a real deal for viewers even outside the forum.

Keeping SOTM is key in my opinion and always been.

Thanks for everyone involved,

Mark


----------



## 31610 (Aug 20, 2017)

Matt man I don’t know what to tell you . I know in January we had the conversation about 3D printed frames and u said your looking for new ideas new ways and new materials for ssotm. That said if u go to a vote will it turn into a show and shine or popularity contest or only 5 people will vote . I might not always agree with you but if it’s not broken don’t fix it . Maybe a couple judges be fine it’s tough call with no cut and dry answer


----------



## MIsling (Sep 7, 2017)

I could go either way on this one, but I voted to not vote.

I have been happy with how you have been running it. I joined after the drama from the last SOTM, but I have been around long enough to enough to feel some of the fallout from it. I would hate to see something like that happen again.

At the same time, I understand the reasoning behind a vote. Technically more fair, and could give a greater degree of involvement and objectivity.

If you get enough interest, a panel of judges seems like a good compromise to me.


----------



## mattwalt (Jan 5, 2017)

The whole point of SOTM is to showcase awesome builds - to to have a platform for frame makers to display their work.

I've used a theme approach so its not simply 'anything' that can be added or nominated - Also made (or even just displayed for the first time publicly) during the month is also fairly important.

It was seeing the works such as Nathan masters and Sharker and Metro Grade Goods and Quercusuber and Mark Seljan and Flipgun and... and... which got me excited about the forum in the first place. It got me making frames and fiddling with rubber. Its allowed me to introduce the sport to a few people.

With my location its unlikely to meet like any of the members of the site - Though Q does live nearby and have met. And we need to get together again soon hopefully and actually get some slinging done. Though I'd still call many members genuine friends. And members who have left are truly missed. Slingnerd would have been a great wingman for SOTM.

Having the process lighthearted and fun is kinda important - and think being able to reward makers work through badges makes it credible. Slingnerd had a monthly challenge but it seemed to fade after a while. Luckily the current crop of members are awesome - and on the whole are laid back. And the quality of work is outstanding - last month as a case in point. I'd happily own any or all of those, no jokes. All were done to a fantastic standard.

I do like Mark's comment about having a front page display of the winning frame - thats a great idea. Will see if it's possible to set that up. I did set up a gallery for the winners - but lets say I've been a bit slack in that regard... And its kinda hidden unless you look for it. A proper showcase would work better.

On the whole - having a SOTM for me is important. Having it last is also important.

Some great comments - I will mull over and see what can be worked with and implemented.


----------



## mattwalt (Jan 5, 2017)

Think in the first SOTM - I gave feedback on every entry. Would it be useful to do this for every month?

It may give some credence to decisions.


----------



## Alfred E.M. (Jul 5, 2014)

Voting was one of the first things I dropped and assumed the burden of a personal judgement.

TBH - some months literally result in ulcers. Weighing up criteria and approaches...

TBH I'm quite a busy boy most of the time - and going the full long haul explanation isn't feasible.

*Matt - each month you sound more stressed by the decision making process, and now you mention physical maladies - running this contest is taking a heavy toll on you. You make heroic efforts, no doubt, but there are consequences.*

*Everyone thrives on approval and praise. We create something but submit it to a binary competition - you win or you don't - decided by one peer who's pushed too close to the rubber room. A winner is announced and that's all folks, and gallop off into next month. I suggest only 4 competitions a year ... Slingshot of the season or quarter. Two pics required for each submission and like now, nominations not required - simply post your slingshot entry. Likely more and better entries and less stress on everyone. Only a few crank out 12 really fine builds a year. *

*We had a voting mechanism before and it was fun to see how you stacked up against master builders like Eric or Tim. I'd favor a rubic judging system and one thru five places, which spreads the soul food around and is just more fun. A panel is a mini bureaucracy, no bueno ... and this is not a good time to use the stolen election arguments ... we need a poll worker (Matt) to stop corruption and bad behavior, but let anyone with the forum benchmark of ten posts cast one vote, and let many voices rate the buffet.*


----------



## mattwalt (Jan 5, 2017)

*A panel is a mini bureaucracy, no bueno *- Not sure actually. If its run badly then it is. But this isn't like its running a dictatorship  I think anyone involved with a judging based system has to be accountable to their approach and be able to vet their decisions without bias. Having a small panel may moderate the moderators 

To be honest - I was well aware of the consequences when I took this on. I'm thick skinned enough to stand by my decisions and as mentioned more than happy to discuss the decisions I made.

Ward - when we discussed your approach some time back I took on board everything we discussed. And your points are valid.

One thing that we can be absolutely sure of in a 'fair' voting scenario is that the blingingest frames are the ones which will be likely to get votes. Regardless of build quality. Unless natty's are on Mark or Q levels you can be assured they won't be voted for. Misling's entry last month was beautiful...

In fact - as a real world example (And this was one I was called out on). There was an entry which was absolutely stunning. It was shiny, technically complex and finished to an extremely high level. On visual appearance it was a clear choice. And would say in a voted platform would have been overall winner. However going in closely the builders approach had allowed for really bad glue lines - gaps between the core and scales. Which is why it was bumped from top place. Granted this is most probably a rare occurrence, voting will be done at face value. But from my experience its builders who been sending out slingmail or have a following that seem to garner the majority of votes - especially over time. And when its like 30-40 individuals doing the voting that bias swings outcomes. It becomes a popularity contest, end of.

Having a quarterly mass SOTM may alleviate that - though have seen before that even with entry-heavy months (40+ builds) the outcomes were still skewed. My only concern here is that interest may wane with it not being more regular.

Having a open public voting base - again the numbers of voters may help mitigate 'corruption', Though we may still only have a small number of voting participants

Allowing a star rating based vote - People are by nature lazy - having them essentially star rate every aspect of each frame though a great idea, I can't see being practical. Having worked on public prize competitions its mental how few entries companies get when it involves anything more than putting a name on something. I'd highly recommend entering any which are complicated - your chances of winning are high. Its possible maybe if only a handful of frames are selected as top tier...


----------



## MOJAVE MO (Apr 11, 2018)

I wonder how many makers submit an entry to hear the applause and too show the Forum peers that they are progressing their efforts and techniques and appreciation of the sport. I think a SOTM badge is a certain sign of amazing skill, but it isn't why I jump in. I get involved for the journey and the motivation to produce my best frame yet. Then I found the joy through my Forum Friends of finding a person to send that frame off too. Maybe you have come Full Circle Matt? Is this process really broken or do you just feel compelled to reach out and confirm that this imperfect process isn't perfect?

Sent from my iPhone using Tapatalk


----------



## mattwalt (Jan 5, 2017)

Thats a great question MO.

I personally think its fine, I took a few weeks to set up the 'rules' of the current SOTM - based on the past ones and implemented them accordingly - having someone to help mediate my choice would be nice - and someone has offered that assistance for this month which is great (they have stepped down from entering)

There have been some members who have requested a vote-in approach due to being 'fair'. Though as mentioned it's about as unfair in practice as could be possible. I'm open to suggestions - and just getting a gauge of the general opinions that members have.

The only 'issue' I can see is that as a sole judge the outcomes may not be acceptable to everyone - and as such the 'blame' falls on me. Some months choices are not simple by any means - esp the last few months where the builds are very close...

Moving forward the decision is ultimately mine. I will make it according to what/how I see fit to maintain a platform I feel works. Feathers could be ruffled - but thats life


----------



## Pebble Shooter (Mar 29, 2014)

(...) "Though we may still only have a small number of voting participants"."

Not necessarily, Matt, at least according to the number of views into the thousands for every SOTM that takes place. Even if only 10% of all those people (members and non-members) cast a vote every month, it would generate an objective outcome based on a reasonable voter sample, as opposed to the rather subjective and biased result linked to one sole (stressed out ?) SOTM judge. As already mentioned, I would assume that enabling non-forum members to cast a vote may also encourage them to actively join the forum as members = a stronger forum, and increased sales potential for vendors.

Precise guidelines in terms of technical evaluation criteria for slingshots submitted, and from which perspective photographs of submitted work should be taken and presented, should make it possible to get objective votes at the end of the month. Indeed, some voters may be carried away by the stunning polished shine of ill-fitted woodwork, but chances are that many will take the list of evaluation criteria seriously and cross the boxes accordingly: active participation, which is motivating!

Moreover, as with the February outcome, there should be 3 winners (like in the Olympics) i.e. gold, silver, and bronze badges every month, not just when there are numerous entries.

At the end of the day, it's a case of trying something new, and reversing to long-term accustomed procedures if the new approach fails after one or two months.

It will be whatever the majority decides, I guess. Fine by me.


----------



## MakoPat (Mar 12, 2018)

MOJAVE MO said:


> Set it up like judging a boxing match. 3 judges. Points per round both for and against the frame entry per judge. Judge anonymously. Add the points. Or maybe a cage fight to decide the winner!?
> Sent from my iPhone using Tapatalk


I was coming to say something like this. A complex system is more prone to breakdowns. There is a reason simple tribunals are still widely accepted forms of adjudication.

Matt, having all the responsibility is unbalanced, especially for him. Ere'body chiming in DID end the OG SSOTM, mostly due to the bickerin aboot who likes who ans not fairplay.

So I am pro Grading Rubric for the anonymous tribunal.

A simple system for 3 randomly selected (likely by MattWalt since he has been doing a fine job already.) SSF members communicating with the Game Keeper, a.k.a. Matt, would provide maximum balance of fairplay, responsibility, and FUN. Fun being the main FUNction of Flippery.


----------



## mattwalt (Jan 5, 2017)

Pebble Shooter said:


> (...) "Though we may still only have a small number of voting participants"."
> 
> Not necessarily, Matt, at least according to the number of views into the thousands for every SOTM that takes place. Even if only 10% of all those people (members and non-members) cast a vote every month, it would generate an objective outcome based on a reasonable voter sample, as opposed to the rather subjective and biased result linked to one sole (stressed out ?) SOTM judge. As already mentioned, I would assume that enabling non-forum members to cast a vote may also encourage them to actively join the forum as members = a stronger forum, and increased sales potential for vendors.
> 
> ...


The number of voters even if public is simply uncertain - which was my point.

3 winners - some months there simply are not enough high quality frames to justify more than 1. Thats already burnt me big time. Yes its totally subjective on adding.


----------



## Valery (Jan 2, 2020)

The idea of "three side referees" is not bad, I think, but randomly selected judges should not have a decisive vote, only an advisory one. The final decision must remain with the organizer of the competition.


----------



## mattwalt (Jan 5, 2017)

Ere'body chiming in DID end the OG SSOTM, mostly due to the bickerin aboot who likes who ans not fairplay.

LOL - was insane. The fighting, abuse language, moderators being threatened physically... Bans being dished out like KFC. Screenshots taken of discussions... Surprised some members survived 

There is a reason those discussions have been deleted / or turned off from public view.

When I first took on SOTM a member's final act before leaving was to send me the transcripts of some - and said good luck.


----------



## mattwalt (Jan 5, 2017)

Valery said:


> The idea of "three side referees" is not bad, I think, but randomly selected judges should not have a decisive vote, only an advisory one. The final decision must remain with the organizer of the competition.


This is my thinking exactly. Essentially just to moderate the final decision. Which would fall upon one sole individual.


----------



## Ordo (Feb 11, 2018)

Years ago I participated in a gastronomy forum. Every month there was a challenge with specific topics (products, techniques, culinary schools, countries, etc.). The winner had to choose the consequent theme and became the undisputed judge of such competition. In general it worked very well. Just another idea to add to the present confusion.


----------



## skarrd (Jul 20, 2011)

i vote no vote,but am good either way,the three panel /judges sounds interesting tho


----------



## Tree Man (Jun 30, 2016)

flipgun said:


> Stand your ground! It's your bat, your ball and it is in your yard. A voting system eventually leans into fans of,,,?... carrying the game.
> Better the decision of an individual than the disputes of the hoi polloi however good-natured.


Here here!


----------

