I think I’ve already mentioned everything necessary in the README of the github page
1 Like
Novel type of failure:
scrape_memrise.py -m -d “C:\Users\jakub” Chronology of Ancient History - by azrael42 - Memrise
Caught exception
‘NoneType’ object has no attribute 'string
The same error returned with this course:
Where exactly did this happen? It’s a big course and I didn’t get it at least on the first level.
EDIT: Nevermind, I see what the problem is. The script is not able to handle empty multimedia levels. I didn’t know those even existed. I’ll fix it soon.
EDIT2: Okay, you can download the script again from the development branch like you did before.
1 Like
Excellent, this works now. Thanks!
I wonder, would you ever in future consider making a script that would convert Memento <-> Anki ?
Since Anki uses Python, my uneducated guess is that it should be somehow possible to make an Anki deck out of a Memento course (or vice versa)?
After days of scraping, re-checking lengthily, what did not get scraped, etc., to spare others the same suffering, I have the following suggestions for a future version of the scraping scritpt:
1.Check for errors/exceptions BEFORE starting any scraping. (I think at least one of the errors I reported earlier was found out only after loooong time of scraping the given course …)
2. Display only one line of “scraping 12 of 4000” where just the former number would keep changing, instead of listing 4000 lines
3. Display summary of all errors/exceptions together after all courses of a given command line were
4. Offer some parameter that would run a beep after all has been done, or, optionally, another type of beep when error has been encountered.
5. Perhaps also a switch “shut down PC after finished” (makes sense only if a log file is kept somewhere)
processed. Or just write a log file and mention the name and location of the log file as last message.
Explanation:
It is, at least in Win shell, very difficult to scroll through thousands of lines to look up which courses did not get saved successfully and why. And it is not possible at all while the scraping still goes on, even if one is willing to scroll up thousands of lines, the shell window keeps skipping to the new last line…
If at least part of the above suggestions can be implemented, it could save many many hours to someone in future who would be as course-greedy as I was 
For me, enough scraping from tomorrow till my death, perhaps 
What I now dream about is getting the Memento courses converted also for use with Anki which would be handy when travelling (I have Anki on my phone, and it is very handy for online cloud syncing of study progress, and easy to correct wrong flashcards right in the middle of studying them). Opposite conversion would also be usefu (Anki deck to Memento). (And some might wish to scrape from Quizlet to Memento too, though that is perhaps not any similar process)
Is there a way how to upload a Memento course back to Memrise web? You say one cannot create courses, but since I can edit them, I might upload a corrected version of a course.
(Btw, is there no way on Memrise to report a really an entirely erroneous course? Like when most meanings of words are mismatched. Or when the audio is with a non-native accent that messes up phonological oppositions. I have not found a way.)
Another minor issue that might be tackled, unless you already did so:
- Part of course name listed as creator name.
Am not sure if this happens still, but I now noticed while editing a course I scraped with the earliest version of the script, that
the course named “Manual of Standard Tibetan-Audio” created by user KungaSonamLhundrup111, the script mis-created the following course info:
“title”: “Manual of Standard Tibetan”,
“author”: “Audio”
Though this was done by the 5.0 version of Memento and the early script.
Thanks for all your work.
Going now to tackle the cookie thing for one-level courses and that seems to be my last scraping in this life, unless I loose all the date.
Vsičko dobro
I could implement a few of those suggestions, but if you want to shut down your PC, you can just write that as another command after the python command. I believe Windows also uses the ; character for separating commands.
However I do not intend to implement uploading a Memento course to Memrise, because for all I know that may not be possible or at least not practical. Also it would depend on whether you have permission to edit the given on Memrise in the first place. I also don’t intend to make a script for converting Memento → Anki, because I do not use Anki and don’t know much about it. Though I may try to create a script for Anki → Memento.
Technically you can create courses, but you would have to do it manually. I simply meant that the app itself is not capable of creating or editing courses (for now). Nothing is stopping you from manually creating new files in the format of a Memento course that the app can read.
That is not possible because the majority of the problems happen during scraping, especially when I’m downloading every individual item after having checked every level. I could try to skip a certain item if an exception occurs, but then that will create problems later when Memento tries to read the course and I’d rather the scraping just stop so I can take a look at it instead of ending up with an incomplete course which would require another scrape to fix anyway.
When you’re scraping a huge course, I recommend checking it for unusual levels and trying to scrape only them first to check if there are any problems.
I think this is largely unnecessary for now, because there are already plenty of tools for converting Memrise courses to Anki decks, so you could just use those.
Anyways, I just pushed a new version which
- better scrapes the course creator’s name (hopefully)
- plays audio on errors or course complete (if you install the pip module “beepy”)
- writes “scraping item X of N” on the same line, so scraping any course should now output the same amount of lines which means it will be easier to read the console output when scrolling up
By the way, have you tried the Android version? What are your thoughts on it compared to PC?
Haven’t tried the Android. Sadly, there’s next to no memory on my recycled phone, and so I cannot install anything (unless I figure out how to uninstall “compulsory” apps like YouTube etc).
Forgot to report a problem encountered on “Indian Geography” course scraping:
python scrape_memrise.py -m -d “C:\Users\jakub” Indian Geography - by Niran24 - Memrise
Caught Exception ‘List index out of range’
(announced after scraping item 117)
Also forgot to say:
The instruction to how to find the cookie (for use with one-level) which I had from you mentions twice a single quote (’) while it was a double quote ("). Both as the character before which to finish copying and the characters in which to enclose the whole cookie string in the command line.
(Chrome on Win10)
I pushed a new update to the script a few days ago which is newer than what you have. Try that version. If it still fails, at least it will show a more detailed error message.
Hey, sorry for not looking into this sooner. I kinda forgot about it.
The problem was caused because an item of type image did not provide an image url to download. I worked around that so now the script won’t crash in that situation.
Looks interesting enough!
@mementoseeds thanks for this great tool as those courses I am learning will be gone, sad 
…
now I am trying to download them and use on my own computer before it’s too late
after reading the instruction
How to install
The easiest way to install Memento is by downloading a prebuilt version from the releases page. Go to this page, click on the assets dropdown menu and download a version that corresponds to your platform.
I downloaded “Memento_v0.6_Linux_64-bit.7z”, but found there is only 1 file, named “memento”. What to do now with this file? Thank you.
Is this app in the F-Droid app store? I only found a note taking app: