Script for Bulk Upload Audio

I’d like to share my bulk upload audio script that I created. It uploads Google TTS audio to any words that do not have it in your database.

It relies on Python 3 but with only a little bit of programming experience you should be able to get it working. It’s a little bit more user-friendly than other resources out there.

It only works for Chinese but I think you can get it working for other languages with just a little bit of work. Just change the end of the url on line 57 of main.py to something that accommodates your language.

Anyway, here’s the url to the script https://github.com/DrewSSP/bulk-audio-upload

5 Likes

I am trying your script. And to do that, I followed the following steps, which all worked well, until the running of the script.

  1. Download Python 3 from https://www.python.org/downloads/ and install it.
    Version v3.6.1 is the latest (as of April 3, 2017) and is 29 MB. (less than 5 min)
  1. Download the script from https://github.com/DrewSSP/bulk-audio-upload using either step 2a or 2b (less than 1 min).
    2a. Using the ‘Clone or Download’ button, which offers the option ‘Download ZIP’. Extract the ZIP, to place of your choice. For example 'Downloads\bulk-audio-upload-master)
    2.b You can also just download the individual files: ) and place in a directory of your choice.
  1. Open a command prompt: Start -> Run -> cmd (Win7) or Right-click Start -> Comand promopt (Win10)
  2. Change to the directory where you placed the files,
    e.g. >cd C:\Users\my-user\Downloads\bulk-audio-upload-master
  3. Create a cookie file with the name 'variables.py
    e.g. >notepad .\variables.py
    As a preperation, type (or copy-paste) the following:

cookies = {‘name1’:‘value1’,‘name2’:‘value2’
}

  1. Now you need to find the cookies. This is perhaps to more difficult part.
    Using the Chrome browser as an example (not sure exactly how to do on other browsers): (15-30 min)
    6a. Login to Memrise
    6b. Menu (the three dots) -> More Tools -> Developers Tools (CTRL-SHIFT-I). A window will appear at the bottom of the screen. If that window is too small it can be resized by clicking exactly on the border (a two-point arrow will appear) and dragging it up (of left).
    6c. In that new view click the ‘Application’ tab, open Cookies in the left hand pane and select ‘memrise.com’. You need to format them as shown in the example below.
If you know how to format, you can skip the next steps and follow your own method. Otherwise, follow these steps to format the cookies:
6d. To copy the values double click on the name, then right-click and choose 'Copy'. 
6e. Paste onto the spot where it says 'name1' in the text file (variables.py). Before and after the name should be a single quote (').
6f. Double click the value, right-click and again choose 'Copy'. 
6g. Again paste into the text file, but now on the spot 'value1'. Again, before and after the value should be a single quote (').
6h. Do the the same for all the cookies.
* There should always be a colon (:) between the name and the value.
* There should be a comma (,) between each two cookies.
* After the last cookie there should be an closing accolate (}).
6i. After you have done all the cookies, save the file.
Example of the cookies file:
cookies = {'_sp_id.7bc7': '06d67edb75b999999.1466999953.100.1555544393.1234573782',
           '_sp_ses.7bc7': '*',
           'csrftoken': 'nxIto89I10jvMe45lt5xBJ8xnQkWayh3',
           'fbm_143688012353890': 'base_domain=.www.memrise.com',
           'fbsr_143688012353890': 'bGMeJtlEkaCEISsa4th4J1-FvygfuhFgBuu7qnnS1M.eyJhbGdvcml03f6OiJITUFDLVNIQTI1NiIsImNvZGUiOiJBUUNxNTh6ekpDMDlpNGNpTjNyaVRYR3VEN0xHZHM1TEJyMmFCVlJBTmVHeVl4WFlWanM1QVRpMDBtSzFwMWNFV2dxa28wUUxrWUVoT0RKLVdQOG1DdHVncjZhSGdxMVBEaEp3MGhTbV9pRXA3ckcxZU02d2M0LTNubFFCVkVaU0tUVml3eXFKdy1FZmxJYTVuZGJZUlhBXzVlNXdGcFFKWWVJem83ZllwcV9COWExdkJ0S3ZFQlB6OGQ2c2w0azMyOWtxcmFyVjJRYXpodVh3WXVlbVc4ejNlUUc5TE11SXJsakxTbW1hOXN3cUREVzEtcl94WlNLRlRucklsV3FxY3kzN0g1UnFVeTRwOFZ0TVVSXzEzd0Z0TjBrNF9PZjgzalJHYVZjZkV5dWFqUDJLQVhSRkxmT3RrU0R5d3lRSnRncGRMVC1ZZTVRTkNKN0xNRWxfcEVVWSIsImlzc3VlZF9hdCI6MTQ2ODY0Mzk4MiwidXNlcl9pZCI6IjUwMzc3MTY3OCJ9',
           'i18next': 'en',
           'sessionid': 'xrxg3zofonxnfmf5gfdgv5444defa71'}
7. Run the script. 

This step 7 didn’t work for me!
I first tried this way:
"\program files (x86)\Python36-32\python" main.py https://www.memrise.com/course/1441234/my-words-for-language-learning/edit/database/9876543/

Traceback (most recent call last):
  File "main.py", line 1, in <module>
    import requests, tempfile, sys
ModuleNotFoundError: No module named 'requests'

2nd time:

cd "C:\Program Files (x86)\Python36-32"
python "C:\Users\my-user\Downloads\bulk-audio-upload-master\main.py" https://www.memrise.com/course/1441234/my-words-for-language-learning/edit/database/9876543
Traceback (most recent call last):
  File "main.py", line 1, in <module>
    import requests, tempfile, sys
ModuleNotFoundError: No module named 'requests'

I also tried to run the script ‘google’ in the folder ‘scripts’, which worked fine, which told me that somehow python itself was working:
C:\Program Files (x86)\Python36-32\Tools\scripts>…\python .\google.py test
It opened the browser with google.com, searching for the text ‘test’.

In the instruction, you did say a number of libraries are needed. I am not sure how to ‘activate’ or configure these libraries.

What should I do to get the script running?

Thank you!

As you suspected, the issue is with the libraries. If you have pip installed this is very easy to fix. If not, first install pip using the instructions found here https://pip.pypa.io/en/stable/installing/.

Once you have pip installed, just run pip install requests and pip install tempfile and that will likely fix your issue.

[quote=“DrewSSP, post:3, topic:3092”]
If you have pip installed
[/quote]It says on the website you mentioned: “pip is already installed if you’re using Python 2 >=2.7.9 or Python 3 >=3.4 binaries downloaded from python.org, but you’ll need to upgrade pip.” I’ve v3.6.1, so, yes, I have pip.
To upgrade pip I tried:

C:\Program Files (x86)\Python36-32>python -m pip install -U pip
Requirement already up-to-date: pip in c:\program files (x86)\python36-32\lib\site-packages

After this, I tried:

C:\Program Files (x86)\Python36-32>pip install requests
’pip’ is not recognized as an internal or external command,
operable program or batch file.
C:\Program Files (x86)\Python36-32>python pip install requests
python: can’t open file ‘pip’: [Errno 2] No such file or directory

So, Python says that the ‘Requirement’ is ‘already up-to-date’, but pip is not a recognized command.
Note, I work on Windows.

Edit: No, pip is not a command, but a script, that needs to be run by using python. See next post.

The Windows command is slightly different from Linux (which @DrewSSP probably uses for good reasons)

C:\Program Files (x86)\Python36-32>python -m pip install requests
Collecting requests
Downloading requests-2.13.0-py2.py3-none-any.whl (584kB)
100% |████████████████████████████████| 593kB 57kB/s
Installing collected packages: requests
Successfully installed requests-2.13.0

So requests is installed now.
Then tried installing ‘pip install tempfile’ - that still doesn’t work though. This is the output:

C:\Program Files (x86)\Python36-32>python -m pip install tempfile
Collecting tempfile
Downloading Tempfile-0.5.tar.gz
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File “”, line 1, in
File “C:\Users\SUPERU~1\AppData\Local\Temp\pip-build-61v5lvl8\tempfile\setup.py”, line 2, in
raise RuntimeError(“Package ‘Tempfile’ must not be downloaded from pypi”)
RuntimeError: Package ‘Tempfile’ must not be downloaded from pypi>

----------------------------------------

Command “python setup.py egg_info” failed with error code 1 in C:\Users\SUPERU~1\AppData\Local\Temp\pip-build-61v5lvl8\tempfile\a

What should I do now?

Note: you need to be Administrator. I tried first with a standard user (my user accounts are non-admin):

C:\Program Files (x86)\Python36-32\Lib\site-packages\pip>…\python -m pip install requests
Collecting requests
Downloading requests-2.13.0-py2.py3-none-any.whl (584kB)
100% |████████████████████████████████| 593kB 239kB/s
Installing collected packages: requests
Exception:
Traceback (most recent call last):
File “C:\Program Files (x86)\Python36-32\lib\site-packages\pip\basecommand.py”, line 215, in main
status = self.run(options, args)
…(cut out some lines)…
PermissionError: [WinError 5] Access is denied: ‘C:\Program Files (x86)\Python36-32\Lib\site-packages\requests’

If you get this error, run the command line (CMD) as an Administrator.

Have you tried running the script since installing Requests? First try that and let me know if it doesn’t work.

[quote=“DrewSSP, post:6, topic:3092”]
tried running the script since installing Requests
[/quote]Yes, I have. Forgot to include it. As you can see no luck. I suppose because ‘python -m pip install tempfile’ failed. See result here:

C:\Program Files (x86)\Python36-32>python "C:\Users\user\Downloads\bulk-audio-upload-master\main.py" https://www.memrise.com/course/1441234/my-words-for-language-learning/edit/database/9876543

Traceback (most recent call last):
File “C:\Users\user\Downloads\bulk-audio-upload-master\main.py”, line 2, in
from variables import cookies
File “C:\Users\user\Downloads\bulk-audio-upload-master\variables.py”, line 3
ajs_user_id 22147479
^
SyntaxError: invalid syntax

According to the traceback that you sent, the error is not with anything in the main.py file. In fact, it looks like you have all of the dependencies properly installed.

The issue is with your variables.py file. Please make sure that you follow the formatting guidelines for the cookies, as specified in the instructions.

1 Like

That was right: I had forgotten to finish the syntax with a closing accolade (}) and had even some junk notes left! Of course that wouldn’t work.

Now I have removed that obstacle, I tried this:

python “C:\Users\user\Downloads\bulk-audio-upload-master\main.pyhttps://www.memrise.com/course/1441234/my-words-for-language-learning/edit/database/9876543
Traceback (most recent call last):
File “C:\Users\user\Downloads\bulk-audio-upload-master\main.py”, line 4, in
from lxml import html
ModuleNotFoundError: No module named ‘lxml’

This meant that ‘main.py’ needs package lxml, so installed as follows:

C:\Program Files (x86)\Python36-32>python -m pip install lxml
Collecting lxml
Downloading lxml-3.7.3-cp36-cp36m-win32.whl (2.9MB)
100% |████████████████████████████████| 2.9MB 124kB/s
Installing collected packages: lxml
Successfully installed lxml-3.7.3

Unfortunately, now it gives an error with the code itself:

Traceback (most recent call last):
File “C:\Users\Michael\Downloads\bulk-audio-upload-master\main.py”, line 76, in
number_of_pages = int(html.fromstring(requests.get(course_database_url, cookies=cookies).content).xpath("//div[contains(@class, ‘pagination’)]/ul/li")[-2].text_content())
IndexError: list index out of range

Is it that the script can somehow not handle the ‘index range’ for this particular url or course or level? Or could it mean it cannot properly access the memrise url?

I’ve seen this happen before when I was working with the code myself. The most likely issue is that your cookies aren’t matching those that Memrise is expecting to see. Memrise sometimes changes your cookies and this might have happened since you originally found and copied them to your computer.

One way to check this is through the python console. Try the following:

python
import requests
cookies = [[[[PASTE YOUR COOKIES HERE]]]]
r = requests.get("https://www.memrise.com/course/1441234/my-words-for-language-learning/edit/database/9876543", cookies=cookies)
r.text

This will output the HTML of your first database page. Try to read it and search for some of the items in your database. If they’re not there it’s because your cookies are innaccurate and Memrise isn’t letting the script see your database. If you do see your database items then the script itself might have issues.

If you find that your cookies are wrong then the obvious solution is to find your cookies once again and re-copy the cookies for your script.

If you still believe the script has issues then please paste the output of the above code with your reply.

1 Like

[quote=“DrewSSP, post:10, topic:3092”]
r = requests.get(“https://www.memrise.com/course/1441234/my-words-for-language-learning/edit/database/9876543”, cookies=cookies)
[/quote]Thank you!Tried that. This is the output:

‘>>>cookies = [[[[{’_ga’:‘GA1.2.649449312.1491119095’, …}]]]]
’>>>r = requests.get(“https://www.memrise.com/course/1441234/my-words-for-language-learning/edit/database/9876543”, cookies=cookies)
Traceback (most recent call last):
File “”, line 1, in
File “C:\Program Files (x86)\Python36-32\lib\site-packages\requests\api.py”, line 70, in get
return request(‘get’, url, params=params, **kwargs)
File “C:\Program Files (x86)\Python36-32\lib\site-packages\requests\api.py”, line 56, in request
return session.request(method=method, url=url, **kwargs)
File “C:\Program Files (x86)\Python36-32\lib\site-packages\requests\sessions.py”, line 474, in request
prep = self.prepare_request(req)
File “C:\Program Files (x86)\Python36-32\lib\site-packages\requests\sessions.py”, line 385, in prepare_request
cookies = cookiejar_from_dict(cookies)
File “C:\Program Files (x86)\Python36-32\lib\site-packages\requests\cookies.py”, line 518, in cookiejar_from_dict
cookiejar.set_cookie(create_cookie(name, cookie_dict[name]))
TypeError: list indices must be integers or slices, not list

For the cookies I tried
cookies = [[[[’_ga’: ‘GA1.2.649449312.1491119095’]]]]

But because that didn’t work, just tried one bracket ([), then an accolade and 4 brackets, and then the “cookies =” statement worked.

After the ‘TypeError’, tried to say only ‘cookies’ (not cookies=cookies) or changing the brackets around the cookies - did not work, similar output.

Any suggestions? Sorry for having so many issues!

Those square brackets are there because I dont want your eye to miss that part of the code. The cookies should be identical to the ones you used in your code. In other words, like this

cookies = {'_sp_id.7bc7': '06d67edb75b999999.1466999953.100.1555544393.1234573782',
           '_sp_ses.7bc7': '*',
           'csrftoken': 'nxIto89I10jvMe45lt5xBJ8xnQkWayh3',
           'fbm_143688012353890': 'base_domain=.www.memrise.com',
           'fbsr_143688012353890': 'bGMeJtlEkaCEISsa4th4J1-FvygfuhFgBuu7qnnS1M.eyJhbGdvcml03f6OiJITUFDLVNIQTI1NiIsImNvZGUiOiJBUUNxNTh6ekpDMDlpNGNpTjNyaVRYR3VEN0xHZHM1TEJyMmFCVlJBTmVHeVl4WFlWanM1QVRpMDBtSzFwMWNFV2dxa28wUUxrWUVoT0RKLVdQOG1DdHVncjZhSGdxMVBEaEp3MGhTbV9pRXA3ckcxZU02d2M0LTNubFFCVkVaU0tUVml3eXFKdy1FZmxJYTVuZGJZUlhBXzVlNXdGcFFKWWVJem83ZllwcV9COWExdkJ0S3ZFQlB6OGQ2c2w0azMyOWtxcmFyVjJRYXpodVh3WXVlbVc4ejNlUUc5TE11SXJsakxTbW1hOXN3cUREVzEtcl94WlNLRlRucklsV3FxY3kzN0g1UnFVeTRwOFZ0TVVSXzEzd0Z0TjBrNF9PZjgzalJHYVZjZkV5dWFqUDJLQVhSRkxmT3RrU0R5d3lRSnRncGRMVC1ZZTVRTkNKN0xNRWxfcEVVWSIsImlzc3VlZF9hdCI6MTQ2ODY0Mzk4MiwidXNlcl9pZCI6IjUwMzc3MTY3OCJ9',
           'i18next': 'en',
           'sessionid': 'xrxg3zofonxnfmf5gfdgv5444defa71'}

This type of object is called a dictionary. If your dictionary only has one item then the cookies that you found are not the right ones. Your cookies should at least have one item called csrftoken in them but don’t forget that you need something that looks like the above.

1 Like

I’ve updated the instruction on the script’s page to help you more. They were a bit out of date. Please focus on your cookies. Every one of your problems is related to the cookies and you need to make sure they are perfect. If you have just one oversight in the cookies, your script will surely fail.

1 Like

@John_Baite Memrise does not have any cookies titled _ga but Github.com does. Your cookies are for Github.com. Go back to the instructions and make your cookies again, step-by-step. I put a little reminder at the end for people who might make the same mistake you do.

1 Like

Right! Tried now an accolades at the beginning and the end - and it worked at the first try. I.e. the ‘r = requests.get…’ worked without errors. And I thinking all along that I should leave those out, and they were only for the file format, not for the command :confounded: .
Now I have to update the cookies itself - you are right - to make 100% sure they are exactly the same as on Memrise. Let you know the outcome. And will update the original ‘instruction’.

Hi @DrewSSP @John_Baite ,

I have my own similar PHP script to bulk upload mp3s to my courses. I haven’t used it in a few months and now I notice it has stopped working. I’m wondering if Memrise has changed their ajax. Do you know if the python script still works?

Thanks!

I’m sorry, I haven’t used it in a couple months either and I can’t really test it right now because of problems with my VPN. I do hope nothing changed.

1 Like

@guestgulkan I didn’t have time either to test this. When I have time, I’ll share the results.

1 Like

Hi @DrewSSP @John_Baite ,

Thanks Guys!

Would be possible to use this script to bulk add audio from forvo into my Hebrew-english course ?

https://www.memrise.com/course/2113613/eteacher-rosen-modern-hebrew-1-2-3/

Thanks,