Multi tool use

Selenium and Java: How do I get all of the text after a WebElement

I am coding a program in Java using WebDriver and am having a little bit of trouble getting the text after the select webElement.

The HTML code for the part of the website that I want is as follows:

<div id="content">


<div id="Pagination"></div>
<div id="mid">
</div>
</div>


The textbox class codes for a search bar and a drop down bar of languages

My Java code is currently able to open chrome using the chrome driver and is able to type into the search bar. I am however not able to get the text that results from the entry.

Image

In the image here, I entered "avoir" into the search bar, and I want all of the text inside the boxes after which do not seem to have any id's or names to be used inside the xpath.

Can someone please help me in finding how to get and save the text from those fields after the dropdown language menu?

Thank you in advance!

The code I have so far:

//import statements not shown
public class WebScrapper {
public WebScrapper() {

}

public WebDriver driver = new ChromeDriver();

public void openTestSite() {

driver.navigate().to(the URL for the website);
}


public void enter(String word) {

WebElement query_editbox =
driver.findElement(By.id("query"));
query_editbox.sendKeys(word);
query_editbox.sendKeys(Keys.RETURN);

}

public void getText() {
//List<WebElement> searchResults =
driver.findElements(By.xpath("//div[@id='mid']/div"));
// Writer writer = new BufferedWriter(new
OutputStreamWriter(new FileOutputStream("status.txt"),
"utf-8"));
//int index = {0};

WebElement result=driver.findElement(By.id("mid"));
System.out.println(result.getText());
}

public static void main(String args) throws IOException {
System.setProperty("webdriver.chrome.driver", "chromedriver");
System.out.println("Hello");

WebScrapper webSrcapper = new WebScrapper();
webSrcapper.openTestSite();
webSrcapper.enter("avoir");
webSrcapper.getText();
System.out.println("Hello");

}


}

2 Answers
2

I have specified three approaches to extract the text from the result box. Please check all the approaches and use the required approach.

If you want to extract all the text, then you can find the element of the result box and then you can get the Text from that.

WebElement result=driver.findElement(By.id("mid"));
System.out.println(result.getText());


If you want to extract the Text based on the Section by section, then you can go with the below approach,

List<WebElement> sectionList=driver.findElements(By.xpath("//div[@id='mid']/div"));
int i=0;
for(WebElement element:sectionList){
System.out.println("Section "+i+":"+element.getText());
i++;
}


If you want to extract the text from specific section, then you can do with the below approach

List<WebElement> sectionList=driver.findElements(By.xpath("//div[@id='mid']/div"));
int i=0;
//Inorder to get the Section 3 Content
int section=2;
for(WebElement element:sectionList){
if(section==i){
System.out.println("Section "+i+":"+element.getText());
}
i++;
}


Edit: To address followup question

I would suggest to use some explicit wait after doing some action which resulting in some element rendering. In your code, after doing some modification, I am getting the result as expected.

openTestSite

enter

getText

Code:

openTestSite Method:

public void openTestSite() {

//driver.navigate().to(the URL for the website);
driver.get("https://wonef.fr/try/");
driver.manage().window().maximize();
//Explicit wait is added after the Page load
WebDriverWait wait=new WebDriverWait(driver,20);
wait.until(ExpectedConditions.titleContains("WoNeF"));
}


enter Method:

public void enter(String word) {

WebElement query_editbox =
driver.findElement(By.id("query"));
query_editbox.sendKeys(word);
//AutoComplete is happening even after sending the Enter Key.
// So, Value needs to be selected from the autocomplete
WebDriverWait wait=new WebDriverWait(driver,20);
wait.until(ExpectedConditions.visibilityOfElementLocated(By.xpath("//div[@class='autocomplete']/div")));
List<WebElement> matchedList=driver.findElements(By.xpath("//div[@class='autocomplete']/div"));

System.out.println(matchedList.size());
for(WebElement element : matchedList){
if(element.getText().equalsIgnoreCase(word)){
element.click();
}
}
//query_editbox.sendKeys(Keys.RETURN);
}


getText Method:

public void getText() {
WebDriverWait wait=new WebDriverWait(driver,20);
wait.until(ExpectedConditions.visibilityOfElementLocated(By.xpath("//div[@id='mid']/div")));
WebElement result=driver.findElement(By.id("mid"));
System.out.println(result.getText());
}


I have tested with the above modified code and it is working fine.

In order to inspect the relevant results for your query, common strategy would be to load a list of search results:

List<WebElement> searchResults = driver.findElements(By.xpath("//div[@id='mid']/div"));


Now you can use stream to iterate over the list and extract your relevant text, by getting the text from child elements of each result:

int index = {0};
searchResults.stream().forEach(result -> {
System.out.println("Printing query result of index: " + index[0]);
result.findElements(By.xpath(".//*")).stream().forEach(webElement -> {
try {
System.out.println(webElement.getText());
} catch (Exception e) {
// Do nothing
}
});
index[0]++;
});


And you would get the output:

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.