Increase randomness in reviewing sessions

Memrise Bug Control Web Bugs
1

It has already been noted in several other places that Memrise doesn’t randomize enough. For example, see Bad randomization of buttons in typing tests and Review is not random.

The places where randomness could be increased include:

  • The distractors in the MC-questions are often the same (and in the same order).
  • The (blue) keys in the typing tests are always displayed in the same order.
  • The correct keys to press in the typing tests often seem to be located at the end of the list.
  • When reviewing, the questions are displayed almost exactly in the order you learned them. For example, in the HSK course the words are learned in alphabetical order. Now, when (say) 50 words are due for review, I first get a series of words starting with b, then a series starting with j, etc.

I think this lack of randomness is harmful for learning, because instead of learning the words, I’m (subconsciously) learning patterns. You can probably find some scientific papers to back this up… :slight_smile:

It should be easy to improve this, by shuffling things around a little. For example, when calculating the due date for an item, the software could randomly add or substract a small amount of time to the exact date.

11 Likes
2

I remember noting this on the old fora, but never got any answer to this. Indeed. I hope it gets solved, especially bullet point 3 as far as I’m concerned.

3

Yeah, I made a suggestion to add some amount of random jitter to scheduling reviews for the future, proportional to the review period. For example something scheduled for 1 day might have +/-20minutes added randomly, but something scheduled for 90 days might have +/- up to a week added randomly. That would mix up items from different review sessions, and change what order they appear in next.

Lots of people commented on that suggestion, but nobody from memrise did. That was years ago.

2 Likes
4

Alright, let’s try to involve someone from the Memrise team, then. @Joshua, would you like to share your thoughts?

1 Like
5

Honestly, adding that much variation to the review schedule (X days +/- a week) would really annoy me and interfere with the way I prefer to use Memrise.

2 Likes
6

Note that @cos is suggesting to randomly add +/- a several days only for items that have a very long due date, so the variation would be proportional to how far away the due date is. Would that also interfere with how you use Memrise?

@Kaspian, could you perhaps explain in a bit more detail how you use Memrise and why this would interfere?

7

wait until you gather the number of learned items that Kaspian, I, Pdao etc have… then even more randomness in the general behaviour of memrise would really be annoying…

you could propose increased randomness in reviewing as an option, but not as an obligatory feature for all users, even for those who never asked for such randomness

1 Like
8

I doubt that Memrise staff is reading this thread, or has any plans on changing their reviewing process (since it is a fundamental part of their SRS algorithm). But whether they are or aren’t, if it ever came to deciding on how best to spend their scant technical resources, I would much rather they spend time and energy focusing on things that are already broken or in hopeless disarray (such as the numerous database and course creation issues that have already been reported) rather than looking at how to change (and possibly break) something that is currently working. My $0.02

4 Likes
9

Off-topic, but this is what I would like: a wrong answer indicates you find the word difficult, so decrease the spacing interval for that word from then on, to perhaps 90% of its former value, get it wrong decrease it again. Ben said they were working on something like this a year ago now, but apparently that was abandoned for now.

Also have different review intervals for different languages and different users (user set).

Memrise theorizes it gives you the word when you are about to forget it, just at the knife’s edge. The time needed to just about forget a word varies with every word, every language, and every user.

10

A post was split to a new topic: I can’t review course words. Reason is SCRIPT FAIL

11

I’m not quite sure what you mean here, could you explain that a bit more? What kind of randomness is annoying?

To be clear: what I am noticing that in my reviewing sessions the words quite often appear in (almost) the same order as I learned them, which I think is bad for proper memorization. There are many ways to improve this, such as adding a little variance proportional to the due dates (as discussed before).
Another, very simple, solution would be to just shuffle the words ‘due today’ - so if you have 60 words due today in a course, they appear in completely random order. That would already make me happy :slight_smile:

12

@robbertbrak,
I am a long-term user, this is what I mean, I have enough items to water/review. Did you see my profile? I did see yours, you’re a beginner :grin:

first of all: in the courses I’m learning I do have enough randomness (think about courses in which you learned well above 200 items, maybe above 1000 items). What would extra-randomness add to this? for items i get wrong, I have the “difficult items” feature

I have thousands of items to review every day (i cannot review everything in one day…) More randomness than this?

as Pdao said, I’d like soooo much just a functional Memrise, with its basic features as they were/are…

(edit: i use memrise solely in its web-version, and for a fair number of languages)

13

For most courses, I water by level. I prefer all the words in one level to come due at the same time, regardless of how long ago I planted them. If there was a week of wiggle room, even for items at the 180 day mark, I might have to wait days for all the items in the lesson to come due before reviewing the lesson. I prefer the predictability of words coming due on a known schedule. Memrise has enough quirks and inconsistencies without intentionally adding more…

I have not noticed this at all. As far as I can tell, I see words in a different order every time.

One question: Are you using the web or the app? I’m using the web, and as far as I can tell, I see words in a different (and random) order every time.

Perhaps the lack of word order randomness is a problem with the app?

1 Like
14

One big problem when you have learned a large number of items is that they start bunching up. If you miss reviewing for a few days, then come back and catch up, you may have accumulated a lot more items than you get in the usual day. But because they all then get scheduled for a fixed time interval from when you reviewed them, they will now all remain bunched up!

For example, lets say you had 20 on Monday, 20 on Tuesday, 20 on Wednesday, and 20 on Thursday - but you were traveling that week and didn’t get a chance to do any memrise. You come back on Friday and review 100 words - they 80 waiting from earlier in the week, plus 20 more that were due Friday.

If all of those were words you knew well, and were already at 180-day interval, all 100 of those words that were previously spread out over a whole week are now all scheduled for exactly the same day in the future! Half a year from now, even if you’re not traveling for a few days and let five days of memrise review accumulate, you’ll still get all 100 words on the same day.

Over time, this effect is self-reinforcing. Every time you miss any days on memrise, words accumulate and then get bunched up, and words that are at the same review level then stay on the same day forever on. There is no countervailing effect that will ever spread them out again. Older words, the ones you’ve known for the longest time, are the most likely to have gotten bunched up in this way, and they will remain in very large clumps.

Those of us who have been using memrise for years have seen this. A particular course may you’ve finished years ago may remain quiet for a month or two and then all of a sudden a couple hundred words are up for review on the same day.

Adding a random interval that’s a small proportion of the scheduled review interval ensures that review words keep getting spread out. It counteracts the bunching-up effect, and keeps the overall equilibrium where if you learn gradually, you also review things gradually, a few each day, not all at once.

4 Likes
15

That is not how spaced repetition should work so this is not a valid argument here (against jitter).

1 Like
16

the absolutely perfect argument of Kaspian is “I am the master of my own learning”… I absolutely agree with her in this respect

2 Likes
17

As long as I keep roughly in step with the extended review periods, I don’t worry too much. As others have said, we must keep control of our own learning.

I know that I am stating the blindingly obvious but isn’t it the case that the intended rigid SRS ‘breaks down’ anyway if either (a) you fail to review items when they fall due (for example, as described above by cos) or (b) when you have learned so many things over a long period of time that the number of items falling due for review on a particular day simply becomes too many to review (for example, as described above by Hydropterre). I don’t worry if, on some days, I don’t review all the items showing as due.

If we all keep learning new things at a high rate and don’t delete any courses from our dashboards from time to time, a point will be reached where there would not be enough hours in the day to do everything.

I don’t think there is going to be a tidy solution to this.

1 Like
18

Well, the solution to that is the “fully learned items” in a separate pool proposal, that I also made years ago, at the same time as I suggested adding some random jitter. But Memrise never commented on that, either - though it did get hundreds of votes on the uservoice forum and spend many months on the first screen of top suggestions.

19

Actually, my motivation for watering by level is precisely to make sure I review based on the spaced repetition system. I regularly have 3000-10,000 items due for watering. Last time I checked, Memrise prioritizes watering based on which items have been due for review longest. The only way I can guarantee that I’m watering the words I know least well is to water by level, starting with lessons planted recently. I’d rather let a bunch of words learned 2 years ago sit for 2-6 weeks than neglect something I planted last week and don’t know well yet.

Or, stated another way: two weeks is 1/12 of 180 days, but is more than double 6 days. Neglecting old stuff keeps the review timing much closer to the SRS than neglecting new stuff. And I’m not going to sacrifice learning new stuff just to keep up with old stuff coming due for review.

That I prefer to water all the items in a lesson together is primarily a matter of convenience… it’s annoying to have a few items scattered here and there that come due at different times. But convenience versus annoyance matters. More convenience = more fun => more time spent learning => more learning. More annoyance = less fun, less time spent, and less learning.

Edit: Just to be clear and to tie this in to the earlier topic on jitter - I don’t always neglect the older stuff. I don’t want Memrise to add randomness to the review schedule, because sometimes (especially with Turkish), I do want to water a lesson at the 180 day mark as soon as it comes due.

3 Likes
20

One possible solution is to water 40 today, 40 tomorrow, 40 the next day, and so on, until you’ve caught back up again, assuming you continue to have 20 more words coming due for review per day. Sure, 180 days from now you’ll get 40 due per day instead of 20, but it would keep them more spread out than watering all 100 in a single day would.

There’s nothing particularly sacred about reviewing at intervals 48, 96, and 180 days… Once you reach that point in the SRS, you can vary your review schedule by several days (or even weeks) without harming your learning.

1 Like