One thing that I think worked well was that Mac let people know their stories were still being considered after an initial reading pass.
There were over 300 stories subbed for the Absolute Visions anthology. When you're reading for an anthology you have not only the usual "is this a publishable story" question, you have the issue of stories that will work together as an anthology, which in some respects is a bit like assembling a playlist.
The stories need to work together, and then they need to be in a particular order.
This is something I was going to ask about. I'm glad you posted this. I think once we decide how we'll select stories we should be able to tell folks their submission is still in play.
When I helped start a creative magazine at a college many years ago, the way we selected was based on votes among the group, then the numbers were whittled down. It was almost like a tournament. Each day we voted a submission up or voted one down. That's fine when you're in a room full of people and everybody is looking at the same piece of paper. On an online format such as this, something else needs to be done.