Export lists from databases / courses I have created?

Wow! Thank you so much for your effort! I tried it quite a few times and waited patiently but in the end I always just got a blank tab. :sweat_smile:
Thank you very much once again, that is a big help!
neko-chan

1 Like

I think the greatest thing you have done is, that it indeed doesn’t need any technical knowledge, that scripts usually require. Thank you!
It only copies the primary columns I understand. Any chance of improving it, exporting all columns, or alternatives, or/and the audio files (or at least the audio file names; if there were another script for exporting the files, a list with the names is helpful in matching and renaming). E.g. if we use an URL like this:
https://www.memrise.com/course//edit/# would.

1 Like

Thank you! I’m not the author of the script, but I suspect that it would not be possible to download sound files, as it would require you to log in. At the moment, the script does not require you to log in, but I have submitted a feature request to the author, which you can track here: https://github.com/nad2000/memrise-scraper/issues/2

Concerning exporting all columns, could you send me an example of a course that does have more than two columns? I might not be able to work on it right now as I have exams coming up, but I will do what I can.

Actually, most courses have more than 2 columns, but only the creator and contributors can see them when going to the edit mode. Those extra columns I meant. I do not have any course with three columns.

BTW: if the script runs on the web-server, then yes, logging in is a different deal. However, if the script runs in the browser, the browser has already the login cookie if the user is logged on, and the problem should be solved by itself. But who knows, their is somewhere a catch - I am not a script writer.

Not my idea, but something I got from someone else on an other thread that I can’t find now (I had copied the code into a Google doc). This has worked very well for me to export my words in a way that I can copy it straight into Quizlet:

Export Memrise course words to CSV.

  1. Log into memrise.com
  2. Navigate to course home page (e.g. http://www.memrise.com/course/335725/comprehensive-german-duolingo-vocabulary/)
  3. Open Developer Console
    To open the developer console window on Chrome, use the keyboard shortcut Ctrl Shift J (on Windows) or Command Option J (on Mac). Alternatively, you can use theChrome menu in the browser window, select the option “More Tools,” and then select “Developer Tools.”
  4. Paste below script and hit enter
  5. After all urls have been fetched, copy final word list into spreadsheet.

SCRIPT:

(() => {

function getWords(courseId, level) {
const url = https://www.memrise.com/ajax/session/?course_id=${courseId}&level_index=${level}&session_slug=preview
console.log('Fetching words from ’ + url)
return fetch(url, { credentials: ‘same-origin’ })
// parse response
.then(res => {
return res.status === 200
? res.json()
// map results
.then(data => {
return data.learnables.map(row => ({
original: row.item.value,
translation: row.definition.value
}))
})
.then(words => {
return getWords(courseId, level + 1)
.then(words.concat.bind(words))
})
: []
})
.catch(err => {
console.error(err)
return []
})
}

// fetch
const start = 1
const courseId = location.href.slice(30).match(/\d+/)[0]
getWords(courseId, start)
// format as csv
.then(words => {
console.log(words.length + ’ words’)
return words.map(word => word.translation + ‘\t’ + word.original + ‘\n’).join(’’)
})
// print
.then(console.log)

})()

Hey, if you’re still looking for a solution, perhaps my exporter will work for you. It should export all of the columns, but you’ll need to download it and right now it’s for Windows only.

Thanks a lot!
It wasn’t working until I downloaded the new version:

but, it does not support Arabic. I get these characters in the csv file :frowning:
ط§ظ„ط§ط³طھط¹ط§ط°ط© ظˆط§ظ„ظپط§طھط­ط© ظٹظژظˆظ’ظ…ظگ ط§ظ„ط¬ظژط²ظژط§ط،ظگ ظˆظژط§ظ„ط­ظگط³ظژط§ط¨ظگ (4){ظٹظژظˆظ’ظ…ظگ ط§ظ„ط¯ظگظ‘ظٹظ†ظگ} ط§ظ„ظپط§طھط­ط©

UPDATE:
Actually, the issue is not Arabic-specific…This is what happened after scrapping a course written in French:
anti-agrأ©gants plaquettaires
The “é” is not recognized

Did you open the file in Excel? Open it in notepad and see if it looks correct there.

I believe the proble with Excel is this - https://www.webtoffee.com/how-to-save-csv-excel-file-as-utf-8-encoded/

Let me try to fix this in the exporter itself.

1 Like

This is what I see if I open the csv in notepad:

I believe it looks fine there. In order to open it in Excel, try the steps in the link above (and don’t forget to change the separator from , to ; on one of the screens during the process.

I’ll keep looking into this.

1 Like

Seems to not be working for large courses. Timeout after a while.

Can you give me an example?

PS E:\Downloads> PS E:\Downloads> .\MemriseScraper.exe 1209
Get-Process : A positional parameter cannot be found that accepts argument ‘.\MemriseScraper.exe’.
At line:1 char:1

  • PS E:\Downloads> .\MemriseScraper.exe 1209
  •   + CategoryInfo          : InvalidArgument: (:) [Get-Process], ParameterBindingException
      + FullyQualifiedErrorId : PositionalParameterNotFound,Microsoft.PowerShell.Commands.GetProcessCommand
    
    

PS E:\Downloads> …

… : The term ‘…

…’ is not recognized as the
name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was
included, verify that the path is correct and try again.
At line:1 char:1

  • … …
  •   + CategoryInfo          : ObjectNotFound: (.................................:String) [], CommandNotFoundException
      + FullyQualifiedErrorId : CommandNotFoundException

Same happens with course 1214.

1 Like

PS E:\Downloads> .\MemriseScraper.exe 1209

Unhandled Exception: System.AggregateException: One or more errors occurred. —> System.Net.WebException: Timeout exceeded while getting response —> System.TimeoutException: The operation has timed out.
at <StartupCode$FSharp-Core>[email protected](Boolean _arg4)
at Microsoft.FSharp.Control.AsyncPrimitives.CallThenInvokeNoHijackCheck[a,b](AsyncActivation1 ctxt, FSharpFunc2 userCode, b result1)
at Microsoft.FSharp.Control.Trampoline.Execute(FSharpFunc2 firstAction) --- End of inner exception stack trace --- at [email protected](Exception _arg2) at <StartupCode$FSharp-Core>[email protected](Exception exn) at Microsoft.FSharp.Control.AsyncPrimitives.CallFilterThenInvoke[T](AsyncActivation1 ctxt, FSharpFunc2 catchFilter, ExceptionDispatchInfo edi) at Microsoft.FSharp.Control.Trampoline.Execute(FSharpFunc2 firstAction)
— End of stack trace from previous location where exception was thrown —
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at Microsoft.FSharp.Control.AsyncResult1.Commit() at Microsoft.FSharp.Control.AsyncPrimitives.RunSynchronouslyInCurrentThread[a](CancellationToken cancellationToken, FSharpAsync1 computation)
at Microsoft.FSharp.Control.AsyncPrimitives.RunSynchronously[T](CancellationToken cancellationToken, FSharpAsync1 computation, FSharpOption1 timeout)
at Microsoft.FSharp.Control.FSharpAsync.RunSynchronously[T](FSharpAsync1 computation, FSharpOption1 timeout, FSharpOption1 cancellationToken) at [email protected](HtmlNode w) at Microsoft.FSharp.Primitives.Basics.List.map[T,TResult](FSharpFunc2 mapping, FSharpList1 x) at [email protected](String level) at Microsoft.FSharp.Collections.ArrayModule.Parallel.Map@1324-3.Invoke(Int32 i) at System.Threading.Tasks.Parallel.<>c__DisplayClass17_01.b__1()
at System.Threading.Tasks.Task.InnerInvokeWithArg(Task childTask)
at System.Threading.Tasks.Task.<>c__DisplayClass176_0.b__0(Object )
— End of inner exception stack trace —
at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)
at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken)
at System.Threading.Tasks.Parallel.ForWorker[TLocal](Int32 fromInclusive, Int32 toExclusive, ParallelOptions parallelOptions, Action1 body, Action2 bodyWithState, Func4 bodyWithLocal, Func1 localInit, Action1 localFinally) at System.Threading.Tasks.Parallel.For(Int32 fromInclusive, Int32 toExclusive, Action1 body)
at Microsoft.FSharp.Collections.ArrayModule.Parallel.Map[T,TResult](FSharpFunc`2 mapping, T[] array)
at Program.main(String[] argv)

It worked for 1209 on the first try for me. If you have a shaky connection or Memrise’s API hiccups, it could crash the whole export because I did not bother with retries.

I uploaded the CSV if you need it: https://filebin.ca/4aOPzHQHBx3a/1209_2019-03-16.csv

Thanks for the file.

I don’t know what the problem is but it fails every time. Today I tried 16 times with course 1214 without luck. I even opened serveral instances and staggered the download to see if they would stop at the same time, they didn’t. some failed early while others continued. This rules out connection problems.

Hi! Have you tried data-miner.io?

It’s a Chrome extension, very easy to use, no need code skills. Free the first 500 credits per month. Each month your count goes back to 0 and you get another 500 credits.

I have managed to export all my courses in order to import to Anki.

2 Likes

Thank you!

1 Like

Hey, could you update your website? This very helpful tool ain’t working no more.

3 Likes

I just checked, and the Memrise Scraper tool is working again. It appears to do the job perfectly.

I’ve noticed that with larger courses ( > ~3000 items) a spurious error message pops up, but the processing of the course continues correctly. After several minutes, click on the browser “go back” arrow, then click on the file name that’s shown after "This course has already been scraped and can be found here: … ".

@tech189 - thank you for repairing your Raspberry Pi system!

3 Likes