Pandas apply on custom function results in segmetnation fault


Pandas apply on custom function results in segmetnation fault



I am applying a custom method on a dataframe using the apply method. When a dataframe with more than 2 rows (tuples) are passed, it results in the kernel being terminated (dead) in the jupyter notebook. When running the same on terminal, it results in a segmentation Fault.



The method works for an individual row or for 2 rows, but not more than that. Both the calls below works with the custom function myTrial.


myTrial


myTrial(pd.ix[3,:])
newPD2 = pd.head(2).apply(myTrial, axis=1)



But this results in the following error.


newPD2 = pd.head(3).apply(myTrial, axis=1)
The kernel appears to have died. It will restart automatically.



The method myTrial uses alignment function pairwise2.align.globalmx from BioPython and other inbuilt python functions. I am providing the function below:


myTrial


pairwise2.align.globalmx


BioPython



I am having a dataframe with 10,000 rows and 8 columns. I am suing a server with 256 GB RAM.



The function is as follows


from Bio import pairwise2
def myTrial(pdf):
source = pdf['source']
targ = pdf['target']

if source == targ:
pdf['sourceAlign'] = source
pdf['targetAlign'] = source
pdf['joint'] = source

return pdf

alignments = pairwise2.align.globalmx(source, targ,1,-0.5)
summaDict = dict()
for item in alignments:
lenList = list()
i = 0
while i < len(item[0]):
con = 0
while item[0][i] == item[1][i]:
con += 1
i += 1

if con == 0:
i += 1
else:
lenList.append((con,item[0][i-con:i],item))
con =0

summa = 0
for thing in lenList:
summa += (thing[0]*thing[0])
try:
summaDict[summa].append(lenList)
except:
summaDict[summa] = list()
summaDict[summa].append(lenList)
stuff = sorted(summaDict.keys(),reverse=True)[0]

if len(summaDict[stuff]) > 1:
print(source,targ,summaDict[stuff])

words = summaDict[stuff][0][0][2]

jointWord = ''
for inda in range(len(words[0])):
if words[0][inda] == words[1][inda]:
jointWord += words[0][inda]
else:
if words[0][inda] != '-':
jointWord += 'DEL('+words[0][inda]+')'
if words[1][inda] != '-':
jointWord += 'INS('+words[1][inda]+')'

pdf['sourceAlign'] = words[0]
pdf['targetAlign'] = words[1]
pdf['joint'] = jointWord

return pdf



The dataframe is as follows


type | source | props | target | subtype | p0 | p1 | p2 | p3 | p4
0 | ADJ | najprzytulniejszy | [NEUT, INS, SG] | najprzytulniejszym | NaN | NEUT | INS | SG | None | None
1 | ADJ | sadystyczny | [MASC, DAT, SG] | sadystycznemu | NaN | MASC | DAT | SG | None | None
2 | V | wyrzucić | [FUT, 2, SG] | wyrzucisz | NaN | FUT | 2 | SG | None | None
3 | N | świat | [ACC, SG] | świat | NaN | ACC | SG | None | None | None
4 | N | Marsjanin | [INS, PL] | Marsjanami | NaN | INS | PL | None | None | None



The console output isnt any helpgul either:


[I 19:16:45.709 NotebookApp] Kernel restarted: 604e9df5-6630-4a12-9c13-e9d7a4835da2
[I 19:17:00.710 NotebookApp] KernelRestarter: restarting kernel (1/5)
WARNING:root:kernel 604e9df5-6630-4a12-9c13-e9d7a4835da2 restarted



What can be the possible reason?





One advice: The .ix method is deprecated since 0.20 (pandas.pydata.org/pandas-docs/stable/generated/…).
– Caio Belfort
Jun 30 at 14:42





1 Answer
1



The Bio.pairwise2.globalmx function is causing a segfault, which is out of Pandas' control. Please see Biopython pairwise2 for non-ASCII strings for the solution on how to fix the underlying segfault.






By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

-

Popular posts from this blog

How to input without newline? (Python)

How to input without newline? (Python) I am trying to create a simple file editor in Python via the console (called PyCons). Here is my current program: def PyCons(f): file = open(f, "r") appe = open(f, "a") print("n=====nPyCons Editor, v1.2nPython 3.6.1n") global i i = 0 for line in file: i += 1 print(" {}| ".format(i) + line, end='') for k in range(10000000000000): print("n {}| ".format(i+1), end='') inp = input("") if inp == "@PyCons.save()": print("n=====") break else: i += 1 appe.write("n" + inp) I used the end method to make sure that the existing code in the file was printed properly in its line format, but the input function does not have an end attribute. So, when I go to enter code in the editor: end input end PyCons("magic.html") ... ===== PyCons Editor, v1.2 Python 3.6.1 1| <p>Hello!...
Read more

C++ thread error: no type named ‘type’ MINGW

C++ thread error: no type named ‘type’ MINGW I'm currently doing a project for my operational systems class and I have to make a program to find prime numbers to run in threads. So I did this: #include <iostream> #include <cmath> #include <thread> #define THREADNUMBER 100 using namespace std; int CONTADOR = 0; int CONTADORTHREADEXECUTADA = 0; int primeRange(int v1, int v2) { int a, limit; bool isprime; for (int i = v1; i <= v2; i++) { if (i % 2 != 0 && i % 3 != 0) { isprime = true; limit = i / 2; if(i == 1) isprime =false; limit = (int)sqrt(i); //General case for(a=2; a <= limit; a++){ if(i % i == 0 && i != 2){ isprime = false; break; } } if (isprime) { CONTADOR++; } } } CONTADORTHREADEXECUTADA++; return 1; } int main(int argc, char *argv[ ] ) { int number1 = atoi(argv[1]); int num...
Read more

Analog for TagView in flutter

Image
Analog for TagView in flutter I want to have UI like this: With native Android it can be done with libraries like https://github.com/whilu/AndroidTagView, how can it be done with flutter? 2 Answers 2 you can make the same UI by combining the Wrap and Chip widget as @wasyl montioned . but this is a full example about what you need Wrap Chip Notes : spacing deleteIcon avatar Chip Shape new Column( crossAxisAlignment: CrossAxisAlignment.stretch, children: <Widget>[ new Wrap( children: <Widget>[ Chip( label: new Text("Java"), onDeleted: () {}, labelPadding: EdgeInsets.all(2.0), deleteIcon: Icon(Icons.clear), ), Chip( label: new Text("C++"), onDeleted: () {}, labelPadding: EdgeInse...
Read more