Wow! Thank you so much for your effort! I tried it quite a few times and waited patiently but in the end I always just got a blank tab.
Thank you very much once again, that is a big help!
neko-chan
I think the greatest thing you have done is, that it indeed doesn’t need any technical knowledge, that scripts usually require. Thank you!
It only copies the primary columns I understand. Any chance of improving it, exporting all columns, or alternatives, or/and the audio files (or at least the audio file names; if there were another script for exporting the files, a list with the names is helpful in matching and renaming). E.g. if we use an URL like this:
https://www.memrise.com/course//edit/# would.
Thank you! I’m not the author of the script, but I suspect that it would not be possible to download sound files, as it would require you to log in. At the moment, the script does not require you to log in, but I have submitted a feature request to the author, which you can track here: https://github.com/nad2000/memrise-scraper/issues/2
Concerning exporting all columns, could you send me an example of a course that does have more than two columns? I might not be able to work on it right now as I have exams coming up, but I will do what I can.
Actually, most courses have more than 2 columns, but only the creator and contributors can see them when going to the edit mode. Those extra columns I meant. I do not have any course with three columns.
BTW: if the script runs on the web-server, then yes, logging in is a different deal. However, if the script runs in the browser, the browser has already the login cookie if the user is logged on, and the problem should be solved by itself. But who knows, their is somewhere a catch - I am not a script writer.
Not my idea, but something I got from someone else on an other thread that I can’t find now (I had copied the code into a Google doc). This has worked very well for me to export my words in a way that I can copy it straight into Quizlet:
Export Memrise course words to CSV.
- Log into memrise.com
- Navigate to course home page (e.g. http://www.memrise.com/course/335725/comprehensive-german-duolingo-vocabulary/)
- Open Developer Console
To open the developer console window on Chrome, use the keyboard shortcut Ctrl Shift J (on Windows) or Command Option J (on Mac). Alternatively, you can use theChrome menu in the browser window, select the option “More Tools,” and then select “Developer Tools.” - Paste below script and hit enter
- After all urls have been fetched, copy final word list into spreadsheet.
SCRIPT:
(() => {
function getWords(courseId, level) {
const url = https://www.memrise.com/ajax/session/?course_id=${courseId}&level_index=${level}&session_slug=preview
console.log('Fetching words from ’ + url)
return fetch(url, { credentials: ‘same-origin’ })
// parse response
.then(res => {
return res.status === 200
? res.json()
// map results
.then(data => {
return data.learnables.map(row => ({
original: row.item.value,
translation: row.definition.value
}))
})
.then(words => {
return getWords(courseId, level + 1)
.then(words.concat.bind(words))
})
: []
})
.catch(err => {
console.error(err)
return []
})
}
// fetch
const start = 1
const courseId = location.href.slice(30).match(/\d+/)[0]
getWords(courseId, start)
// format as csv
.then(words => {
console.log(words.length + ’ words’)
return words.map(word => word.translation + ‘\t’ + word.original + ‘\n’).join(’’)
})
// print
.then(console.log)
})()
Hey, if you’re still looking for a solution, perhaps my exporter will work for you. It should export all of the columns, but you’ll need to download it and right now it’s for Windows only.
Thanks a lot!
It wasn’t working until I downloaded the new version:
but, it does not support Arabic. I get these characters in the csv file
ط§ظ„ط§ط³طھط¹ط§ط°ط© ظˆط§ظ„ظپط§طھطط© ظٹظژظˆظ’ظ…ظگ ط§ظ„ط¬ظژط²ظژط§ط،ظگ ظˆظژط§ظ„طظگط³ظژط§ط¨ظگ (4){ظٹظژظˆظ’ظ…ظگ ط§ظ„ط¯ظگظ‘ظٹظ†ظگ} ط§ظ„ظپط§طھطط©
UPDATE:
Actually, the issue is not Arabic-specific…This is what happened after scrapping a course written in French:
anti-agrأ©gants plaquettaires
The “é” is not recognized
Did you open the file in Excel? Open it in notepad and see if it looks correct there.
I believe the proble with Excel is this - https://www.webtoffee.com/how-to-save-csv-excel-file-as-utf-8-encoded/
Let me try to fix this in the exporter itself.
This is what I see if I open the csv in notepad:
I believe it looks fine there. In order to open it in Excel, try the steps in the link above (and don’t forget to change the separator from , to ; on one of the screens during the process.
I’ll keep looking into this.
Seems to not be working for large courses. Timeout after a while.
Can you give me an example?
PS E:\Downloads> PS E:\Downloads> .\MemriseScraper.exe 1209
Get-Process : A positional parameter cannot be found that accepts argument ‘.\MemriseScraper.exe’.
At line:1 char:1
- PS E:\Downloads> .\MemriseScraper.exe 1209
-
+ CategoryInfo : InvalidArgument: (:) [Get-Process], ParameterBindingException + FullyQualifiedErrorId : PositionalParameterNotFound,Microsoft.PowerShell.Commands.GetProcessCommand
PS E:\Downloads> …
…
… : The term ‘…
…
…’ is not recognized as the
name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was
included, verify that the path is correct and try again.
At line:1 char:1
- … …
-
+ CategoryInfo : ObjectNotFound: (.................................:String) [], CommandNotFoundException + FullyQualifiedErrorId : CommandNotFoundException
Same happens with course 1214.
PS E:\Downloads> .\MemriseScraper.exe 1209
…
Unhandled Exception: System.AggregateException: One or more errors occurred. —> System.Net.WebException: Timeout exceeded while getting response —> System.TimeoutException: The operation has timed out.
at <StartupCode$FSharp-Core>[email protected](Boolean _arg4)
at Microsoft.FSharp.Control.AsyncPrimitives.CallThenInvokeNoHijackCheck[a,b](AsyncActivation1 ctxt, FSharpFunc
2 userCode, b result1)
at Microsoft.FSharp.Control.Trampoline.Execute(FSharpFunc2 firstAction) --- End of inner exception stack trace --- at [email protected](Exception _arg2) at <StartupCode$FSharp-Core>[email protected](Exception exn) at Microsoft.FSharp.Control.AsyncPrimitives.CallFilterThenInvoke[T](AsyncActivation
1 ctxt, FSharpFunc2 catchFilter, ExceptionDispatchInfo edi) at Microsoft.FSharp.Control.Trampoline.Execute(FSharpFunc
2 firstAction)
— End of stack trace from previous location where exception was thrown —
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at Microsoft.FSharp.Control.AsyncResult1.Commit() at Microsoft.FSharp.Control.AsyncPrimitives.RunSynchronouslyInCurrentThread[a](CancellationToken cancellationToken, FSharpAsync
1 computation)
at Microsoft.FSharp.Control.AsyncPrimitives.RunSynchronously[T](CancellationToken cancellationToken, FSharpAsync1 computation, FSharpOption
1 timeout)
at Microsoft.FSharp.Control.FSharpAsync.RunSynchronously[T](FSharpAsync1 computation, FSharpOption
1 timeout, FSharpOption1 cancellationToken) at [email protected](HtmlNode w) at Microsoft.FSharp.Primitives.Basics.List.map[T,TResult](FSharpFunc
2 mapping, FSharpList1 x) at [email protected](String level) at Microsoft.FSharp.Collections.ArrayModule.Parallel.Map@1324-3.Invoke(Int32 i) at System.Threading.Tasks.Parallel.<>c__DisplayClass17_0
1.b__1()
at System.Threading.Tasks.Task.InnerInvokeWithArg(Task childTask)
at System.Threading.Tasks.Task.<>c__DisplayClass176_0.b__0(Object )
— End of inner exception stack trace —
at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)
at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken)
at System.Threading.Tasks.Parallel.ForWorker[TLocal](Int32 fromInclusive, Int32 toExclusive, ParallelOptions parallelOptions, Action1 body, Action
2 bodyWithState, Func4 bodyWithLocal, Func
1 localInit, Action1 localFinally) at System.Threading.Tasks.Parallel.For(Int32 fromInclusive, Int32 toExclusive, Action
1 body)
at Microsoft.FSharp.Collections.ArrayModule.Parallel.Map[T,TResult](FSharpFunc`2 mapping, T[] array)
at Program.main(String[] argv)
It worked for 1209 on the first try for me. If you have a shaky connection or Memrise’s API hiccups, it could crash the whole export because I did not bother with retries.
I uploaded the CSV if you need it: https://filebin.ca/4aOPzHQHBx3a/1209_2019-03-16.csv
Thanks for the file.
I don’t know what the problem is but it fails every time. Today I tried 16 times with course 1214 without luck. I even opened serveral instances and staggered the download to see if they would stop at the same time, they didn’t. some failed early while others continued. This rules out connection problems.
Hi! Have you tried data-miner.io?
It’s a Chrome extension, very easy to use, no need code skills. Free the first 500 credits per month. Each month your count goes back to 0 and you get another 500 credits.
I have managed to export all my courses in order to import to Anki.
Thank you!
Hey, could you update your website? This very helpful tool ain’t working no more.
I just checked, and the Memrise Scraper tool is working again. It appears to do the job perfectly.
I’ve noticed that with larger courses ( > ~3000 items) a spurious error message pops up, but the processing of the course continues correctly. After several minutes, click on the browser “go back” arrow, then click on the file name that’s shown after "This course has already been scraped and can be found here: … ".
@tech189 - thank you for repairing your Raspberry Pi system!