Friday, March 11, 2016

How to debug neovim python remote plugin

I really really really want to have debugger integration with my Vim setup and while the plugins for old Vim were a little wacky, the new architecture of NeoVim seems promising, so I decided to give lldb.nvim a go.

It didn't work. This is a (epic boring)  story of how I debugged and fixed the issues.

Step 1: update

Update your neovim to the latest release to avoid fighting issues that have already been solved. At the time of writing, I used:
  • nvim 0.1.2 from Homebrew
  • OS X 10.10.5
  • XCode 7.0
  • lldb-340.4.70

Step 2: Diagnose

PyThreadState_get error

If you're on OS X, chances are you have more than one Python version installed and that's where the trouble comes from. If you get this error message

>>> import lldb
Fatal Python error: PyThreadState_Get: no current thread

it's most likely because you're trying to import a module that has been linked with a different version of Python. The lldb module comes with the XCode developer tools and was linked with the default system version of Python which lives in (remember this)

/System/Library/Frameworks/Python.framework/Versions/2.7/bin/python

so this is the python version you should use to run the lldb.nvim remote plugin. On my system, BTW, lldb module lives in

/Applications/Xcode.app/Contents/SharedFrameworks/LLDB.framework/Resources/Python

Step 3: Install neovim Python module

The neovim module has probably already been installed with neovim but perhaps not in the correct Python version. You can try to  import neovim in the system Python. If it fails, you'll need to install it using easy_install or pip:

sudo /System/Library/Frameworks/Python.framework/Versions/2.7/bin/python -m easy_install neovim

This will install the neovim package into the system Python distribution (needs sudo) using the easy_install tool.

Step 4: Configure neovim to use the system Python

In your neovim config file, add this line:

let g:python_host_prog = '/System/Library/Frameworks/Python.framework/Versions/2.7/bin/python'

This ensures that neovim will start the system Python (which has access to lldb and neovim modules) to host the plugin. After this, you should be all set!

Step 5: Using $PYTHONPATH?

If you do use $PYTHONPATH with your non-system Python, you'll have trouble as well. Before launching the system Python from nvim, you'll need to clean this variable otherwise the packages will interfere with the system Python's packages.

I do that using a small wrapper script ~/syspython2 which gets invoked from nvim as the g:python_host_prog

#!/bin/sh
# running the OS X system python. Required to import the lldb module.
export PYTHONPATH="/Applications/Xcode.app/Contents/SharedFrameworks/LLDB.framework/Resources/Python"

# enable these for debugging
#echo "--" >> ~/syspython2.log
#echo "$@" >> ~/syspython2.log
/System/Library/Frameworks/Python.framework/Versions/2.7/bin/python "$@"

Step 6: Diagnose

Still having trouble?

  • Don't forget to run :UpdateRemotePlugins
  • Enable logging in the ~/syspython2 script
  • Check using pstree | less if neovim is launching the correct Python binary
  • Double-check you can import neovim and lldb modules from the system Python
  • Make sure lldb.neovim is installed correctly - the file lldb.nvim/rplugin/python/lldb_nvim.py must exist
  • NeoVim also tries to load Python 3 plugins, you may need to do the same for Python 3
  • Try to debug /usr/local/Cellar/neovim/0.1.2/share/nvim/runtime/autoload/remote/host.vim  using debugging vim methods
  • More info about lldb Python module here on StackOverflow

67 comments:

  1. Thanks a million trillion times! I was getting suicidal,
    Here's one hundred smilies:
    :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :)

    ReplyDelete
  2. lldb works smooth with the default macOS python (/usr/bin/python). Which is also used in LLDB Python Reference: http://lldb.llvm.org/python-reference.html
    But there is a need to install all the dependencies and even pip itself for this version. My workaround was:
    1) installing pip with /usr/bin/python using get-pip.py (no luck with easy_install):
    download https://bootstrap.pypa.io/get-pip.py
    run the downloaded: sudo /usr/bin/python get-pip.py
    2) installing the 'six' module (sudo /usr/bin/python -m pip install six)
    3) installing neovim, but 'pip install' for each dependency:
    sudo /usr/bin/python -m pip install greenlet
    sudo /usr/bin/python -m pip install msgpack-python
    sudo /usr/bin/python -m pip install futures
    sudo /usr/bin/python -m pip install trollius
    sudo /usr/bin/python -m pip install neovim
    4) adding 'export PYTHONPATH=/Developer/Library/PrivateFrameworks/LLDB.framework/Resources/Python' to my ~/.bash_profile
    5) setting nvim's python to /usr/bin/python
    let g:python_host_prog = '/usr/bin/python'
    But this is clean Python on my system, all the modules that I previously installed is in /System/Library/Frameworks/Python.framework/... so it's not so great.

    ReplyDelete
    Replies
    1. Sorry, PYTHONPATH should be set as:
      PYTHONPATH=/Applications/Xcode.app/Contents/SharedFrameworks/LLDB.framework/Resources/Python

      Delete
  3. Thanks for your great tutorial. I had problem with addressing python host like this:
    let g:python_host_prog = '~/syspython2'
    This should be:
    let g:python_host_prog = '/Users/username/syspython2'
    With this configuration it works :)

    ReplyDelete
  4. Thank you. Excited to use lldb in neovim now!

    ReplyDelete
  5. This comment has been removed by the author.

    ReplyDelete
  6. Nice knowledge gaining article. This post is really the best on this valuable topic.
    Data Science Course in Bangalore

    ReplyDelete
  7. I’ve read some good stuff here. Definitely worth bookmarking for revisiting. I surprise how much effort you put to create such a great informative website.
    Data Science Training in Bangalore

    ReplyDelete
  8. It is the level of shared understandings required in our most inconsequential forms of social interaction which necessarily require that individuals assume complicated shared knowledge that is too complex for even the must sophisticated forms of artificial intelligence as conceived to date, in which propositions are either true or false and premises must follow deductively. artificial intelligence training in hyderabad

    ReplyDelete
  9. This comment has been removed by the author.

    ReplyDelete
  10. really awesome post this is. Truly, one of the best posts I've ever witnessed to see in my whole life. Wow, just keep it up.Learn best Business Analytics Course in Hyderabad

    ReplyDelete
  11. This is actually the data I'm searching for, I was unable to have requested a less difficult perused with extraordinary tips this way... Much appreciated!
    artificial intelligence course in delhi

    ReplyDelete
  12. I feel very impressed by the information shared and I really learned a lot from it thank you.
    Data Science Course in Hyderabad

    ReplyDelete
  13. Big data is on the rise and so is the requirement for professionals with skills. The training enables folk to pursue an interesting career as a Data Scientist. data science course in hyderabad

    ReplyDelete
  14. Awesome blog with excellent information and fantastic writing skills found very useful thanks for Sharing you.Data Analytics Course Online 360DigiTMG

    ReplyDelete
  15. Very informative message! There is so much information here that can help any business get started with a successful social media campaign!

    Business Analytics Course in Bangalore

    ReplyDelete
  16. It was only after the advent of Big Data that the world realized the tremendous force behind Data. This is a completely new profession with promising career growth and immense possibilities. Industries need Data Scientists in a large number, but there aren't enough currently and this is why the opportunities for fresher are almost endless. data science course syllabus

    ReplyDelete
  17. This is an excellent post I seen thanks to share it. It is really what I wanted to see hope in future you will continue for sharing such a excellent post.
    data science institute in hyderabad

    ReplyDelete
  18. First off, data science training helps candidates choose better career paths. Therefore, these jobs are offered across the world. This training will help you be more skillful, proficient and technological aware. data science course syllabus

    ReplyDelete
  19. Great Article. Thank you for sharing! Really an awesome post data science course in Hyderabad

    ReplyDelete
  20. This is a great motivational article. In fact, I am happy with your good work. They publish very supportive data, really. Continue. Continue blogging. Hope you explore your next post
    certification of data science

    ReplyDelete
  21. As always your articles do inspire me. Every single detail you have posted was great.
    data science certification in noida

    ReplyDelete
  22. Your content is very unique and understandable useful for the readers keep update more article like this.
    data scientist course in noida

    ReplyDelete
  23. Happy to visit your blog, I am by all accounts forward to more solid articles and I figure we as a whole wish to thank such huge numbers of good articles, blog to impart to us.
    data scientist course

    ReplyDelete
  24. Really Nice Information It's Very Helpful All courses Checkout Here.
    data science in malaysia

    ReplyDelete
  25. Fantastic blog extremely good well enjoyed with the incredible informative content which surely activates the learners to gain the enough knowledge. Which in turn makes the readers to explore themselves and involve deeply in to the subject. Wish you to dispatch the similar content successively in future as well.

    Data Science Training in Raipur

    ReplyDelete
  26. Fantastic blog with excellent information and valuable content just added your blog to my bookmarking sites thank for sharing.
    Data Science Course in Chennai

    ReplyDelete
  27. I want to leave a little comment to support and wish you the best of luck. We wish you the best of luck in all your blogging endeavours.
    Python Training in Hyderabad
    Python Course in Hyderabad

    ReplyDelete
  28. It took a while to understand all the comments, but I really enjoyed the article. It turned out to be really helpful for me and I'm positive for all the reviewers here! It's always nice to be able to not only be informed, but also entertained! I'm sure you enjoyed writing this article.
    Business Analytics Course

    ReplyDelete
  29. You totally coordinate our desire and the assortment of our data.
    artificial intelligence course in pune

    ReplyDelete
  30. Incredibly conventional blog and articles. I am realy very happy to visit your blog. Directly I am found which I truly need. Thankful to you and keeping it together for your new post.
    best data science course online

    ReplyDelete
  31. Thanks For your post. This is rally helpful for Data science for beginner.
    data science course in aurangabad

    ReplyDelete
  32. First You have a great blog .I will be interested in more similar topics. I see you have really very useful topics, I will be always checking your blog thanks.
    data scientist course in malaysia



    ReplyDelete
  33. Very good points you wrote here..Great stuff...I think you've made some truly interesting points.Keep up the good work.
    data science course

    ReplyDelete
  34. I read that Post and got it fine and informative. Please share more like that...
    data scientist training in malaysia

    ReplyDelete
  35. It is imperative that we read blog post very carefully. I am already done it and find that this post is really amazing.
    data science course

    ReplyDelete
  36. Very good message. I came across your blog and wanted to tell you that I really enjoyed reading your articles.
    Data Analytics Bangalore

    ReplyDelete
  37. Very good message. I came across your blog and wanted to tell you that I really enjoyed reading your articles.

    Business Analytics Course in Kolkata

    ReplyDelete
  38. Your content is nothing short of brilliant in many ways. I think this is engaging and eye-opening material. Thank you so much for caring about your content and your readers.
    data analytics courses in hyderabad

    ReplyDelete
  39. It's like you've got the point right, but forgot to include your readers. Maybe you should think about it from different angles.

    Business Analytics Course in Gorakhpur

    ReplyDelete
  40. 360DigiTMG is the most recommended Data Science course institute in Chennai. Get trained by top professionals from IIT, IIM and, ISB. Enroll now!
    business analytics course in bhubaneshwar

    ReplyDelete
  41. The increase in big data has led to a boom in the field of Data Science spiking ample career opportunities. Enroll in the Data Science training in Bangalore and invest in emerging skills and transform any business by wrangling, analyzing, and visualizing data. Give your career a makeover and gain in-depth knowledge on how to extract valuable insights from complex and large sets of data. Get to work on a live project which is designed to give hands-on experience to you along with career guidance and mentorship.

    Business Analytics Course in Jodhpur

    ReplyDelete
  42. Register for the Data Science certification in Bangalore and gain recognition and credibility in your organization. Learn the techniques to examine large data sets and discover patterns that are valuable to predict market trends. Learn Data Exploration and Visualizations, Neural Networks and Deep Learning, Model Evaluation and Analysis. This course will ensure that you are challenged to go from a beginner with no Data Science experience to someone who can juggle data with ease.

    Data Science Course in Bangalore with Placement

    ReplyDelete
  43. 360DigiTMG provides you the best Data Science Course in Bangalore, with excellent training from the best trainers in the field and real-time projects, soon you will be an expert in the domain with the highest paid job.


    Best Data Science Training institute in Bangalore

    ReplyDelete
  44. 360DigiTMG is India's number one Data Science Training Institute. Avail of the best training from professional trainers with a world-class curriculum, LMS Access, real-time projects, and assignments that will help you grab your dream job.

    Data Science Course in Bangalore with Placement

    ReplyDelete
  45. K-Nearest Neighbour (KNN) algorithm is one of the uncomplicated ML algorithms based on the Supervised Learning Technique. It assumes the similarities between the new and available data and categorizes them based on pre-existing groups while storing all the available data. KNN incorporates a data point based on similarities of new and old data. It is used for both regression and classification of data, but the latter is used frequently. To learn more about KNN start your Data Science training today with 360DigiTMG

    Data Science Course in Jaipur

    ReplyDelete
  46. Join Data Science classes with 360DigiTMG and get job-ready in 4 months. A world-class curriculum, LMS Access, Real-time projects, and assignments will aid you in getting a high-paying job.

    Business Analytics Course in Jodhpur

    ReplyDelete
  47. Explore what the best Data Science training institute is offering you. Learn advanced technologies from the best industry experts.
    data analytics course in patna

    ReplyDelete
  48. This 1xbet korea is completely different to a sport corresponding to Blackjack where the probabilities change as cards are dealt. With an inside wager, you won’t win as much, however your odds of profitable are much greater. The inside wager choices are sometimes all grouped in a grid in the direction of|in course of} the underside of the desk. When you play Roulette, there are quantity of} other ways you'll be able to|you presumably can} wager. Much of the tablespace is covered with a large grid that outlines the wheel numbers. If you are utilizing a genetic algorithm where a number of the} inhabitants have a score of zero, however proceed to} wish to embody them in the choice course of, you must to} consider using a different choice method.

    ReplyDelete
  49. The blog is very informative and useful.Thanks for the blog.
    Python Course in Nagpur

    ReplyDelete
  50. Thanks for sharing this here. Great Post, really it was very helpful for us. I found this blog to be very useful! python classes in satara

    ReplyDelete
  51. Nice! you are sharing such helpful and easy to understandable blog. i have no words for say i just say thanks because it is helpful for me. Looking for top-notch online tuition for Class 7? Ziyyara Edutech offers the finest home tuition classes for Class 7.
    Book A Free Demo Today visit Home tuition classes for class 7

    ReplyDelete