Language Course Creators: Help Me Test this Forvo Integration Script

Since this post on the old forum is still active, I’m copying it here where we can continue it. Apologies for formatting errors.


Arete_Hime: and here is the thread archived for when the above link stops working: http://web.archive.org/web/20160701105703/http://www.memrise.com/thread/1334937/

Language Course Creators: Help Me Test this Forvo Integration Script

I got tired of going back and forth to Forvo.com when adding audio files to new levels. So, I wrote a user script that does the checking for me. I tried to make it so that it will work for any language, but I can’t test them all myself.

If you are a course creator, do me a favor and install the script to try it out. It doesn’t make any automatic changes to your course, it just displays a new column for you alone. You can decide whether or not to save the MP3 and upload it to your own course.

https://greasyfork.org/en/scripts/6305-forvo-audio-in-the-memrise-level-editor

Posted by carpiediem 11/7/14 , last update Jun 18 (1 week ago)


Please let me know any problems you run into or other feedback you have.

Posted by carpiediem 11/7/14 (1 year ago)


Terrific idea! I tried it on Firefox33.0.3/Win7x64, got a Forvo API, but it is just loading endlessly. The speaker icon (above) never shows up. The Forvo website counts the requests however.

Posted by JamesBP 11/7/14 (1 year ago)


This kind of funcionality would be incredibly useful on the website though. I think users desperately need more automation to create larger (4-5000 word) courses.

Posted by JamesBP 11/7/14 (1 year ago)


James, I’m getting the same result as you. Can’t seem to find the solution yet, but I’ve asked for help and will try again later this week. Thanks for letting me know.

Posted by carpiediem 11/10/14 (1 year ago)


There are several other issues with your code, but the most serious is that you are trying to do a cross-domain request using jQuery. You need to use GM_xmlhttpRequest instead. Read up on “same origin policy”.

Posted by crt32 11/10/14 (1 year ago)


I know about the same origin policy, that’s why I’m using jsonp. I know the code is hacky (it’s difficult to hook into Memrise’s load events), but it does run normally on Chrome.

Posted by carpiediem 11/10/14 (1 year ago)


I seems like in Firefox, you can’t create global JavaScript variables that can be accessed from outside the userscript, thus your callback is undefined (that’s the error I got using your script).

Is there a reason for not using GM_xmlhttpRequest?

Posted by crt32 11/10/14 (1 year ago)


crt32, I has used jQuery because I know it better, it handles JSON very nicely, and it worked in Chrome on almost the first try. I tried out your suggestion though, and it did the trick for both browsers. Thanks.

That unearthed a new problem with Firefox, though. The right click event is handled differently. I can open up the context menu and save the MP3 link in Chrome, but right-clicking immediately closes the drop-down menu in Firefox. For now, users can still save the file with a middle click.

JamesBP, go ahead and update the user script to see if it works for you. It’s not perfect, but it should be functional enough to save a lot of time maintaining courses.

https://greasyfork.org/en/scripts/6305-forvo-audio-in-the-memrise-level-editor

Posted by carpiediem 11/11/14 (1 year ago)


Thanks for the effort! It’s indeed working now with the middle-mousebutton-download method. By the way I’m not a programmer unfortunately (and don’t want to sound like a jerk), but is there a way to maybe eliminate the downloading part? Sort of like a “1-click-add” button, which is how memrise should work in the first place IMHO :smiley:

Posted by JamesBP 11/11/14 (1 year ago)


I’d love to try this out, on Firefox, but I’m going to wait and watch this thread for a bit longer and see if the Firefox wrinkles get sorted out. Thanks for writing it!

Posted by cos 11/11/14 (1 year ago)


Okay, I went ahead and tried it out anyway :slight_smile:

At least on Firefox, it only seems to work when I edit levels, not when I edit the dictionary. Since I tried editing the dictionary first, I thought that it didn’t work at all. But later when I was editing a level I saw the extra column and realized that it does. It would be helpful to add that to the documentation.

Secondly, I have a minor problem that I think I could easily solve by editing the script, but this is not a programming language I know, so I wonder if one of you could give me the line of code I need. My problem is this: I’m making a Hawaiian course and to make typing easier for people, I’m using a backquote instead of the unicode okina character. This is because okina appears very often in Hawaiian, but is unavailable on most people’s default keyboard layouts, and a backquote looks about the same and works as well.

Problem is, of course, that now about half of my words won’t match anything on forvo even if they are there, because the search is using a different spelling. All I need to do is to make this script automatically translate every backquote in the word to a unicode okina, right before this line, I think:

var word = encodeURI( $(this).attr(“data-word”) );

It’s a simple character substitution on the variable “word”, I think. Though maybe there’s some subtlety I’m missing. And of course encodeURI would need to be able to handle okina, but I expect that it can.

Posted by cos 11/11/14 (1 year ago)


cos, I edited the script so that it works in the database editor now. If that’s not what you meant, please point me to a URL where you’re having trouble.

As for the Hawaiian courses, I see two issues, but both are solvable. Let’s look at the word kaa as an example. The Forvo API will only respond with an audio file if the request for this word is formatted kaa. You’re right that a different punctuation mark is used, but the levels I’ve looked at also include the article he in front of every word: “he kaʻa”. What I’ve done in this version of the script is remove "he " from the front of every Hawaiian word and manually changed the punctuation as necessary. This means that learners would see “he” but not hear it. It’s also possible that it introduced some new weird bugs, since I don’t know how the Hawaiian language works. Let me know if it’s working okay for you, otherwise we can try something else.

Posted by carpiediem 11/12/14 (1 year ago)


oy, this pseudo-markup language is strange. We need a preview option.

Posted by carpiediem 11/12/14 (1 year ago)


I’m not sure what levels you looked at, but my course is “incomplete” and does not show up on searches, so I think you were probably not looking at my course :slight_smile: I definitely make a point to introduce each word on its own, with no determiners (he, he mau, ke, ka, nā), the first time. After a word is introduced I use it in phrases, but I don’t expect to find my phrases on forvo. Just the individual words is good enough to start with :slight_smile:

Yes, I was aware of the weird effect of using backquotes in memrise forum comments, which is why I avoided typing these words in here directly :slight_smile: Anyway, I’ll try your new version of the script now. I’m only going to look for bare words, so I won’t benefit from the he-removal feature you added, but maybe someone else will.

However, removing the okina is actually a bug. Forvo does in fact have words properly spelled with okina in them. What you may not have been aware of is that the okina is not punctuation, but a meaningful consonant. If you remove it, you will significantly change the pronunciation of some words, and often turn them into different words. For example, “kou” is a one syllable word meaning “your”. But if you add an okina between the o and u, you turn it into a two-syllable word meaning “my”. So if your script removes the okina, it’ll change “my” back to “your”, and give people an incorrect audio file.

Posted by cos 11/12/14 (1 year ago)


Here’s an example of a recording on forvo with a real unicode okina: http://www.forvo.com/word/Ê»umi/

Posted by cos 11/12/14 (1 year ago)


Huh, memrise mangled that one. Maybe I need to re-encode URL-encoded things here… Let’s see if this works: http://www.forvo.com/word/ko`u/

Posted by cos 11/12/14 (1 year ago)


Right. The script is replacing two apostrophe variants with % 60, the URL version of the okina.

Posted by carpiediem 11/12/14 (1 year ago)


Ahh, cool. I was possibly misled by the formatting oddness of the comment.

Unfortunately, now that I’ve downloaded and installed the new version, I no longer get the extra audio column with forvo lookup, either when editing the database or when editing by level.

Posted by cos 11/12/14 (1 year ago)


Did you try refreshing the editor page once or twice? I don’t know why this works, but it sometimes does for me. I’ve only ever had to do it after installing a new user script, never again after the first use.

Posted by carpiediem 11/12/14 (1 year ago)


Yes, the first time I installed the script I did have to do a page reload to get it working. This time, however, even opening the course in a brand new tab and going to edit didn’t get it working.

I’m probably not reading your code very well because I’ve never written a userscript or used jquery, but do the .find(4)'s imply that the script expects its column to be the 4th pool-column-foo? I’ve already got four: Hawaiian, English, Audio, and Synonyms. (but maybe that has nothing to do with the problem)

Posted by cos 11/12/14 (1 year ago)


Would you mind adding me as a contributor so I can test directly on your course? You can remove me one this is fixed.

Posted by carpiediem 11/12/14 (1 year ago)


I’ve added you.

Posted by cos 11/12/14 (1 year ago)


My fault. I somehow added an extra semicolon to version 0.5 after testing and that was enough to trigger a syntax error. Should be fixed now.

By the way, you’re almost right about the find(4) code; it’s inserting a new column after the 4th column (really the 5th column, since javascript is 0-indexed).

Posted by carpiediem 11/12/14 (1 year ago)


Excellent! It works again, and it found some words with okina in them.

Sadly the Hawaiian audio database on forvo is a bit sparse, but maybe Iʻll be able to find a Hawaiian speaker to contribute some more.

Posted by cos 11/12/14 (1 year ago)


Ok, this script is by far the best that happened on Memrise, Maybe the memrise team can develop it a little bit to integrate it in the interface? I also hope to get “Options” when there are more than one version of the audio, as well as a possibility of “1 click does it all”

Thanks again for the great work

Posted by moe85 11/15/14 (1 year ago)


moe85, the current script will show up to 4 MP3 files if Forvo has them.

JamesBP, I definitely tired to find a way to add audio to Memrise with one click, but it just isn’t possible with client-side code (due to cross-domain security rules). It should be possible if the Memrise developers were to add this sort of functionality themselves.

Posted by carpiediem 11/16/14 (1 year ago)


Installed it successfully on Chrome 38.0.2125.122/OS X Yosemite 10.10 and I can see the check Forvo column and it shows when words are on Forvo, but I can’t actually hear any of them when I click/hover/do anything over the speaker icon. Also agree that Memrise should build this function in and glad you introduced me to Forvo!

Posted by kvsiano 11/16/14 (1 year ago)


kvsiano, the Forvo links won’t work the same as the Memrise ones. Because of the way that Forvo makes the MP3 files available, they can’t be played by a mouse hover event, they can only be downloaded. Perhaps it would be better if the Forvo drop-down menu showed disk or file icons instead of speaker icons. That might avoid some of the confusion.

Posted by carpiediem 11/16/14 (1 year ago)


Ahhh okay well then it works perfectly! I agree that another icon would probably be better, since it’s currently the exact same icon that is used to listen to things on Memrise haha. But the more I use this function the more I wish it could be integrated by the Memrise developers!

Posted by kvsiano 11/16/14 (1 year ago)
1.1.2.3.5.8.13

I feel bad for bumping up an old thread, but do you think you could edit the script so that the user can make it block words to include in the search? For instance, if you choose to block BEGINNING "el " with a space, your script would search for “papel”, not “el papel”. Forvo doesn’t have articles etc. in their vocab mostly, so when I’m using your script to make a course for studying Spanish A1 and A2 levels, it’s not very useful to me unless I edit out all of my articles, refresh, download and upload audio, and hopefully remember which articles go with which word, then edit the words again. It is very useful for languages such as English and Welsh, though.

Posted by 1.1.2.3.5.8.13 6/3/15 (1 year ago)


I never get any hits for the Hebrew course I’m making. I wonder if it’s because I’m putting in words with nikud (vowel markings) and the corresponding forvo items are without. I’ll try some words without nikud that I know are on forvo when I get home.

Posted by cos 6/3/15 (1 year ago)


@fibonacci: Can you add me as an editor in your course so I can see what you mean? That will help me debug this ASAP.

@cos: It ought to be possible to modify the script so that it reads nikud and sends a query to Forvo without them. Can you point me to a list of all the non-English characters that you use for nikud?

Posted by carpiediem 6/4/15 (1 year ago)


Hello

Posted by Charlie20 6/4/15 (1 year ago)


It turns out that’s not the problem, or at least not the sole reason this isn’t working for me. I added “שלום” with no nikud to my database, and “check forvo” found nothing even though I know there are several recordings for it there. However, I just realized I bet the script is checking for the first column, which in my course is English. Hebrew is in the second column.

The script doesn’t know how to read the “test on / prompt with” configuration of the database, does it? IIRC it’s just looking at the first column. Maybe I can remind myself how to edit these scripts and it might be a simple change to just pick which column it looks at.

However, I think the nikud might be an obstacle even if I fix the column issue. I don’t know of a list of all the non-English characters. There are about ten different marks, and most of them can appear on nearly any letter. A couple can only appear on a few letters. So, a list would be long, because it would be the whole alphabet multiplied by like 7 or 8 times.

Posted by cos 6/4/15 (1 year ago)


@cos: I’ll try to work on it this weekend. You’re right that my script probably assumes that columns are in their default positions, but that can be fixed. Hopefully this function will handle the nikud modifications:

http://web.archive.org/web/20120918093154/http://lehelk.com/2011/05/06/script-to-remove-diacritics

Posted by carpiediem 6/4/15 (1 year ago)


What’s everyone talking about

Posted by Charlie20 6/4/15 (1 year ago)


I just saw this existed, followed the steps, and upon reaching the step #4, Plans & Pricing, found out that Forvo no longer offers a free API plan, and what used to be “Free” is now called “Non-Profit”, and costs 1 USD/month. Is the free api key still working for any of you guys who did it earlier?

Posted by loesvanbos 6/4/15 (1 year ago)


I didn’t realize that they stopped accepting new users for the free API, that’s too bad. My API key is still working for free, but it’s limited to 500 queries a day.

Posted by carpiediem 6/4/15 (1 year ago)


On the French courses where I have edit capability, the forvo lookup column is working for me. I haven’t changed the API key in my script since shortly after I first started using it, which appears to have been November.

Posted by cos 6/4/15 (1 year ago)


@carpiediem - did you get a chance to work on it?

Posted by cos 6/8/15 (1 year ago)


@cos, I didn’t yet, but it’s still on my ‘to do’ list. Sorry.

@loesvanbos, it looks like we’ll all be in the same boat soon. I just got this email:

Dear Developer,

This message is a reminder that your Forvo API Key is set to expire in 30 days. Be sure to renew today to keep uninterrupted service for Forvo!

Regards,

Forvo Team. http://api.forvo.com/

Posted by carpiediem 6/9/15 (1 year ago)
_deleted_150823_1550_15

Forvo API
For academic and individual use with Anki, GoldenDict etc
500 requests/day

$1.00 / month

10,000 requests/day
$28.95 / month

Posted by _deleted_150823_1550_15 6/9/15 (1 year ago)


@cos, can you please add me as a contributor to your Hebrew course? Since it’s private, I’m not quite sure what your pronunciations look like. For what it’s worth, the script does seem to work with niqqud vowels if none of the columns have been changed.

Posted by carpiediem 6/29/15 (1 year ago)
Defunctfrenzy

@carpiediem, sorry to jump in on your thread. Is there a way I could contact you? I have an idea for a simple user script but lack the technical know how to create it. Thanks

Posted by Defunctfrenzy 6/30/15 (1 year ago)


Just add @gmail.com to the end of my username. This is also a good place to start: http://wiki.greasespot.net/Tutorials

Posted by carpiediem 6/30/15 (1 year ago)


@carpiediem: Added you as a contributor. http://www.memrise.com/course/702355/hebrew-reading-and-language-intro/

Posted by cos 7/2/15 (12 months ago)


Here’s the issue with making Forvo work in your course: the user script assumes that the word you’re learning (in the foreign language) is in the leftmost column. That’s where Memrise seems to put it by default, so I’m guessing you rearranged things at some point while you were building the course.

It’s easy enough to modify the script to work though, you just need to point it to the correct column in line 36 of the code. For your Hebrew course, you should replace line 36 of the script with this:

var columnNumber = (window.location.pathname=="/course/702355/hebrew-reading-and-language-intro/edit/") ? 2 : 1;

var word = $(this).find(‘td’).eq(columnNumber).find(’.text’).text();

To do the same for other courses, just change the URL that’s being checked and replace the 2 after the question mark with the column you need Forvo to check.

Posted by carpiediem 7/4/15 (12 months ago)


Right - that is what I thought the problem was on Jun 4th. Here’s what I wrote then:

It turns out that’s not the problem, or at least not the sole reason this isn’t working for me. I added “שלום” with no nikud to my database, and “check forvo” found nothing even though I know there are several recordings for it there. However, I just realized I bet the script is checking for the first column, which in my course is English. Hebrew is in the second column.

The script doesn’t know how to read the “test on / prompt with” configuration of the database, does it? IIRC it’s just looking at the first column. Maybe I can remind myself how to edit these scripts and it might be a simple change to just pick which column it looks at.

So, editing to change which column it looks at is simple enough (and you just told us how to do it, thanks), but I still wonder if it could learn to look at the “test on / prompt with” setting and pick the correct column automatically?

Posted by cos 7/4/15 (12 months ago)


The setting you mention doesn’t indicate which is the foreign language. It can be set to either direction. Plus, I think it’s more likely that the “test on” setting will be changed from the default than the column order will be changed from the default.

Posted by carpiediem 7/6/15 (12 months ago)


Does the chrome version work or is it buggy? I’ve tried without results.

Posted by quirkyandsuch 10/24/15 (8 months ago)


Wow this script looks great! I have some questions:

Does it still work? The project is a bit old by now.

When there are several audio files for one word; will I be able to chose the one I like the most? Sadly it can’t be taken for granted that the best file is in the first position.

I’d like to try it out (provided the two above questions can get a “yes”), would somebody care to share their API key with me? I only have a little course and could never use up 500 requests per day.

Posted by Morgengrauen Jun 18 (1 week ago)


Just in case, there is a new forum: http://community.memrise.com/ I think they might shut down this one here and all the content may go down with it. Maybe this tool could have a fresh start in the new forum?

Posted by Morgengrauen Jun 18 (1 week ago)

4 Likes

I’m also keen to use this script but it doesn’t appear to be working… The extra column is not showing up with either Chrome or Firefox.

1 Like

Same problem here. I have the paid API but it’s not working.