UnicodeEncodeError: 'ascii' codec can't encode character u'\xa0' in position 20: ordinal not in range(128)

3.7K Asked by CarolBower in SQL Server , Asked on Jun 4, 2021

Currently I am working on this code:

customer_telno = customer.find('div', 'customer_phone_number')
customer_telno = '' if customer_telno is None else customer_telno.contents[0]
p.customer_info = str(customer_phone + ' ' + customer_telno).strip()

When the above snippet is run I get UnicodeEncodeError :

Traceback (most recent call last):

File "foobar.py", line 792, in 
 p.customer_info = str(customer_phone + ' ' + customer_telno).strip()

UnicodeEncodeError: 'ascii' codec can't encode character u'xa0' in position 20: ordinal not in range(128)

I have tried using everything possible but couldn’t find anything which works consistently.

Can anyone solve this problem?

Answered by Asistha pandey

In case you are facing ordinal not in range 128 error it is because you are converting unicode to encoded bytes using str, so to solve the problem you require to stop str and instead use .encode() to properly encode the strings.

Syntax-

  str.encode(encoding="utf-8",errors="strict")

Also, it returns an encoded version of a string as a bytes object.

Here, an error is set to specify a various error handling scheme and by default, the error is set to “strict” which means that the encoding error raises a UnicodeError, the default encoding is set to “utf-8”.

Using .encode() in your code:

  p.customer_info = u' '.join((customer_phone, customer_telno)).encode('utf-8').strip()

Your Answer

Answer (1)

Ranjana

The UnicodeEncodeError: 'ascii' codec can't encode character u'�' in position 20: ordinal not in range(128) error occurs when Python tries to encode a Unicode string containing characters outside the ASCII range (0-127) using the ASCII codec. The specific character � is a non-breaking space, which is not part of the ASCII character set.

Here's how you can troubleshoot and resolve this error:

Solutions

1. Use UTF-8 Encoding

UTF-8 is a common encoding that can handle any Unicode character.

  # Example: Encoding a string to UTF-8

  unicode_string = u"Some text with non-breaking space�"encoded_string = unicode_string.encode('utf-8')

print(encoded_string)

2. Replace or Ignore Non-ASCII Characters

You can choose to replace non-ASCII characters with a placeholder or ignore them.

  # Example: Replacing non-ASCII charactersunicode_string = u"Some text with non-breaking space�"encoded_string = unicode_string.encode('ascii', 'replace')  # Replaces with '?'print(encoded_string)

  # Example: Ignoring non-ASCII charactersencoded_string = unicode_string.encode('ascii', 'ignore')  # Ignores non-ASCII charactersprint(encoded_string)

3. Using str.encode() with Error Handling

When dealing with file writing or network transmission, you can specify how to handle encoding errors.

  # Example: Writing to a file with UTF-8 encodingwith open('output.txt', 'w', encoding='utf-8') as file:    file.write(unicode_string)# Example: Encoding with error handlingencoded_string = unicode_string.encode('ascii', 'xmlcharrefreplace')  # Replaces with XML character referenceprint(encoded_string)

4. Set the Default Encoding (Not Recommended)

You can change the default encoding in Python, but this approach is generally not recommended because it can lead to other issues.

  import sysreload(sys)sys.setdefaultencoding('utf-8')

Example Scenario

Suppose you are reading a file containing non-ASCII characters and need to process or save the content without encountering encoding errors:

  # Reading from a file and writing to another file with UTF-8 encodingwith open('input.txt', 'r', encoding='utf-8') as infile:    content = infile.read()with open('output.txt', 'w', encoding='utf-8') as outfile:    outfile.write(content)

By explicitly handling the encoding, you can avoid the UnicodeEncodeError and ensure your program processes text data correctly.

Conclusion

The key to resolving UnicodeEncodeError is understanding the encoding requirements of your data and explicitly specifying the appropriate encoding in your code. Using UTF-8 is generally a safe and effective choice for handling Unicode text in Python.

4 Months