| Pages in topic: [1 2 3] > | Failing test translations because of incompetent evaluators Thread poster: Zolboo Batbold
|
|---|
Zolboo Batbold 
Italy
Local time: 14:11
Member (2021)
English to Mongolian
+ ...
SITE LOCALIZER
I did this test translation the other week for a translation agency, it was a Google project. Today I got the feedback from the evaluator and saw my perfectly good translations downvoted, and some dubious, unnatural-sounding translations offered as alternatives. What the .... Shouldn't translation agencies have at least 2-3 evaluators assess the same translation? Well to, you know, get a more objective evaluation? I am fed up with these incompetent evaluators who don't know what they're doing. W... See more I did this test translation the other week for a translation agency, it was a Google project. Today I got the feedback from the evaluator and saw my perfectly good translations downvoted, and some dubious, unnatural-sounding translations offered as alternatives. What the .... Shouldn't translation agencies have at least 2-3 evaluators assess the same translation? Well to, you know, get a more objective evaluation? I am fed up with these incompetent evaluators who don't know what they're doing. What was your reaction to a same problem? How did you handle it? ▲ Collapse
| | | | Samuel Murray
Netherlands
Local time: 14:11
Member (2006)
English to Afrikaans
+ ...
Zolboo Batbold wrote:
I did this test translation the other week for a translation agency, it was a Google project. Today I got the feedback from the evaluator and saw my perfectly good translations downvoted, and some dubious, unnatural-sounding translations offered as alternatives.
This is a risk that comes with the trade.
Shouldn't translation agencies have at least 2-3 evaluators assess the same translation?
I've never heard of such a thing, no.
| | | | mughwI
United States
Local time: 09:11
English to Spanish
+ ...
| Test Reviewers | Oct 20, 2021 |
Let me start by saying that this might not apply in your case. YMMV.
As a long-time LQA consultant, I am often hired to review test translations of potential candidates. Often, the translations are correct but do not follow the strict parameters set by the end client (i.e. expecting the same style/tone used in their arguably badly written website, without actually including this as an instruction).
In other words, the checklists that I have to follow typically contain i... See more Let me start by saying that this might not apply in your case. YMMV.
As a long-time LQA consultant, I am often hired to review test translations of potential candidates. Often, the translations are correct but do not follow the strict parameters set by the end client (i.e. expecting the same style/tone used in their arguably badly written website, without actually including this as an instruction).
In other words, the checklists that I have to follow typically contain information and expectations that were *not* shared with the translator. No glossary, no style guide. Just "do your best".
For my part, I always mention the good points, as well as the clear effort the translator has made given the extremely short deadline, and lack of guidance. AFAIK, the detailed results of my review are not shared with the translator.
Sadly, because of this quirk, great collaboration opportunities are lost along the way. ▲ Collapse
| | | | Zolboo Batbold
Italy
Local time: 14:11
Member (2021)
English to Mongolian
+ ...
TOPIC STARTER SITE LOCALIZER
mughwI wrote:
Let me start by saying that this might not apply in your case. YMMV.
As a long-time LQA consultant, I am often hired to review test translations of potential candidates. Often, the translations are correct but do not follow the strict parameters set by the end client (i.e. expecting the same style/tone used in their arguably badly written website, without actually including this as an instruction).
In other words, the checklists that I have to follow typically contain information and expectations that were *not* shared with the translator. No glossary, no style guide. Just "do your best".
For my part, I always mention the good points, as well as the clear effort the translator has made given the extremely short deadline, and lack of guidance. AFAIK, the detailed results of my review are not shared with the translator.
Sadly, because of this quirk, great collaboration opportunities are lost along the way.
Why are the detailed results not shared with the translator?
| | |
|
|
|
Kevin Fulton
United States
Local time: 09:11
German to English
| Time is money | Oct 20, 2021 |
Zolboo Batbold wrote:
Why are the detailed results not shared with the translator?
Agencies that require test translations often have dozens of tests to assess each year., each requiring both the assessment itself and reporting the result (thumbs up/down). The assessment is for the benefit of the agency, not the candidate. An agency is under no obligation to expend additional time/money on an unsuccessful applicant.
I've done a large number of evaluations over the years and I've often fought the temptation to contact the applicant: "The text has to be completely translated. Leaving sentences in the source language is never acceptable" or "You dimwit! Didn't you look at the accompanying image? The text is about an interior door, not a gate."
My advice is to move on, give no further thought to unsuccessful applications. There are lots of agencies, some with astute assessors.
| | | | Zolboo Batbold
Italy
Local time: 14:11
Member (2021)
English to Mongolian
+ ...
TOPIC STARTER SITE LOCALIZER | Be that as it may | Oct 20, 2021 |
Kevin Fulton wrote:
Zolboo Batbold wrote:
Why are the detailed results not shared with the translator?
Agencies that require test translations often have dozens of tests to assess each year., each requiring both the assessment itself and reporting the result (thumbs up/down). The assessment is for the benefit of the agency, not the candidate. An agency is under no obligation to expend additional time/money on an unsuccessful applicant.
I've done a large number of evaluations over the years and I've often fought the temptation to contact the applicant: "The text has to be completely translated. Leaving sentences in the source language is never acceptable" or "You dimwit! Didn't you look at the accompanying image? The text is about an interior door, not a gate."
My advice is to move on, give no further thought to unsuccessful applications. There are lots of agencies, some with astute assessors.
Thing is, there's a lot of bad translators/reviewers in my language pair (English-Mongolian). I'm not just saying it. It's a fact. There's a lot of misagreements among translators, or even ordinary people, over correct spelling and grammar. It's a constant battle. And there are people who get into the translation business just because they know a foreign language or two.
| | | | Baran Keki
Türkiye
Local time: 16:11
Member
English to Turkish
| They're not 'incompetent' | Oct 20, 2021 |
Zolboo Batbold wrote:
Thing is, there's a lot of bad translators/reviewers in my language pair (English-Mongolian). I'm not just saying it. It's a fact. There's a lot of misagreements among translators, or even ordinary people, over correct spelling and grammar. It's a constant battle. And there are people who get into the translation business just because they know a foreign language or two.
Unfortunately I've had the same problem myself a few times. The fact is a person (translator) who has a financial relationship (a conflict of interest) with an agency cannot be trusted to do an honest evaluation. Pure and simple. There are, of course, good, honest people who go about their task with integrity, but they're in the minority in a language pair such as yours. A PM, not speaking a word of Mongolian, will always take their word over yours. It's dog eat dog when it comes to language pairs where jobs are few and far between.
Agencies should employ third party reviewers/evaluators to assess test translations in their onboarding process, but that never happens.
| | | | mughwI
United States
Local time: 09:11
English to Spanish
+ ...
Zolboo Batbold wrote:
Why are the detailed results not shared with the translator?
No idea. Company policy, maybe?
I was once asked to downgrade a perfectly good translation because the end client thought that grading it as flawless would give the translator ideas about charging more.
I refused, but who knows what they did on their end.
| | |
|
|
|
mughwI
United States
Local time: 09:11
English to Spanish
+ ...
| Lost Opportunity | Oct 20, 2021 |
Baran Keki wrote:
Agencies should employ third party reviewers/evaluators to assess test translations in their onboarding process, but that never happens.
They do. I am one of them. Unfortunately, we often (always?) get outvoted because... reasons. Nothing to do with the actual talent/expertise of the potential candidate.
Of course, there are times when the person applying for a project has more enthusiasm than knowledge, but even then there is a lost opportunity for giving people objective feedback that could steer them in the right direction.
| | | |
If you are objectively sure that your translation was good, then take it as a red flag and be glad you dodged a bullet.
| | | |
After giving it some thought, I think you should thank them for such a short and not so wonderful experience as in the long run you wouldn’t be happy to work with an agency like that. Forget it and move on! Better luck next time…
| | | | | Competent, independent assessors do exist | Oct 20, 2021 |
Baran Keki wrote:.
Agencies should employ third party reviewers/evaluators to assess test translations in their onboarding process, but that never happens.
Apart from when it does.
I’ve assessed tests for agencies I’m now too expensive for. Nine out of ten tests are laughably poor, and I’m sure all nine translators thought they did a really good job.
| | |
|
|
|
Abba Storgen (X)
United States
Local time: 08:11
Greek to English
+ ...
| I've seen worse, and it's a problem far bigger than you think (Part 1) | Oct 20, 2021 |
Such as "eye treatment" been rejected and replaced by "treatment of the eyes"
or the word "primary" rejected because in their opinion it sounds like "primate".
These, for a medical device study, from a well-known LSP's medical department.
Shocked?
If you complain about these to them, you will be blacklisted.
The PM (or anyone else in the entire department) is not a linguist and does not speak the language anyway, and wants as little disruption as possible. In the... See more Such as "eye treatment" been rejected and replaced by "treatment of the eyes"
or the word "primary" rejected because in their opinion it sounds like "primate".
These, for a medical device study, from a well-known LSP's medical department.
Shocked?
If you complain about these to them, you will be blacklisted.
The PM (or anyone else in the entire department) is not a linguist and does not speak the language anyway, and wants as little disruption as possible. In the new paradigm, if you see that an engine is about to fall from the wing, don't say anything. Because the QA department or the project department will be held responsible, and they don't want that. So they will try to contain the problem by silencing you.
You see, in the new culture in some large SLPs, you (as a translator) do not have supporting teams in there. Just potential judges, most if not all of them clueless about the product (but very knowledgeable in office protocols).
The reason they get away with these things is that end-clients rarely check the translation they receive, and even if they do, they don't have the same critical eye as a translator to see the errors or they just say to themselves "they're all the same anyway". That's why a small number of SLPs recently started labeling some projects as "high visibility project", to alarm translators to pay special attention.
This industry's current success is based on that nobody sees the mediocre (or below) results of products sold at a very high price. Also, the enormous pool of desperate people trying to find work. It is not by chance that wherever large unemployment benefits from governments started during COVID, certain translators saw their jobs multiply, because many of the rest didn't need to work that much.
The success will be cut abruptly when Google MT gets a little closer to "acceptable" of what it is now (I give it less than two years). Because it's free. Some very large companies have already started using their own internal MTs anyway, and despite the attempts of SLPs to convince them that they're not good enough, we're not far from them noticing that "when business is back to normal, and the SLP is not afraid of losing the client, the quality of expensive translations is not different than lightly editing our own MT".
That's another reason why I recommend to young people to stay away from this industry, and if they possess a critical mind and at least average talent, to seek careers in industries which value objectivity and transparency. And to quit the "gig industry". At worse, they'll get a more normal working schedule, some time off, health insurance and something tangible to add to their resumes. ▲ Collapse
| | | | | Laughably good | Oct 20, 2021 |
Ice Scream wrote:
Baran Keki wrote:.
Agencies should employ third party reviewers/evaluators to assess test translations in their onboarding process, but that never happens.
Apart from when it does.
I’ve assessed tests for agencies I’m now too expensive for. Nine out of ten tests are laughably poor, and I’m sure all nine translators thought they did a really good job.
Or a laughably good job...
Self-assessment is hard.
Same as for driving. A psychology study found out 88% of respondents thought they were better drivers than average.
Well that beats the average.
---
If the testing process is a failure, better give that agency a miss anyway. Who knows what else they got wrong?
| | | | Baran Keki
Türkiye
Local time: 16:11
Member
English to Turkish
| I hate to bring up the issue of language pairs | Oct 20, 2021 |
I'm sure there are independent, competent and impartial language assessors in major language pairs, especially where the target language is English, and I don't doubt Ice Scream's experience with those candidates as an expensive assessor.
However, the agencies do rely on their existing translators for such tests when the language pair is English to Mongolian, Slovakian, Turkish, Georgian etc. etc.
They don't usually get much work in those language pairs, and unfortunately some of th... See more I'm sure there are independent, competent and impartial language assessors in major language pairs, especially where the target language is English, and I don't doubt Ice Scream's experience with those candidates as an expensive assessor.
However, the agencies do rely on their existing translators for such tests when the language pair is English to Mongolian, Slovakian, Turkish, Georgian etc. etc.
They don't usually get much work in those language pairs, and unfortunately some of those existing translators do not like competition either, if that makes sense.
I remember one occasion where a dear colleague went overboard with sabotaging my test translation and actually gave the game away. They, inevitably, replaced every word with their synonyms when failing to find any translation mistakes and when they ran out of synonyms, they replaced lowercase letters with uppercase letters (like changing "Bank account" (source text) to "Bank Account" (target text)) they did that wherever they could in a 300 word text.
Long story short, I was able to demonstrate this to the Vendor Manager and 'pass' the test.
Not that it did me any good though... That Danish agency started cutting down on rates and introduced longer payment terms as soon as Covid started. Perhaps it's best to give such agencies a miss as others suggested. ▲ Collapse
| | | | | Pages in topic: [1 2 3] > | To report site rules violations or get help, contact a site moderator: You can also contact site staff by submitting a support request » Failing test translations because of incompetent evaluators | Anycount & Translation Office 3000 |
|---|
Translation Office 3000
Translation Office 3000 is an advanced accounting tool for freelance translators and small agencies. TO3000 easily and seamlessly integrates with the business life of professional freelance translators.
More info » |
| | Trados Business Manager Lite |
|---|
Create customer quotes and invoices from within Trados Studio
Trados Business Manager Lite helps to simplify and speed up some of the daily tasks, such as invoicing and reporting, associated with running your freelance translation business.
More info » |
|
| | | | X Sign in to your ProZ.com account... | | | | | |
Login to reply/comment 
Translation news
