How to retrieve tokens by indexing and slicing?
To retrieve tokens by indexing, we need to enter the correct position of the token present in the sentence.
In the sentence assigned to ‘doc5’ the word ‘better’ lies in the third position.Since indexing in python starts from 0,so the index position of the third token will be 2
Similarly slicing can get us a list of tokens by specifying a range of index.
Here, the three tokens from the middle of a sentence has been sliced and also the last four tokens can be implemented in a way mentioned in the snippet above.