Friday, March 11, 2016

How to debug neovim python remote plugin

I really really really want to have debugger integration with my Vim setup and while the plugins for old Vim were a little wacky, the new architecture of NeoVim seems promising, so I decided to give lldb.nvim a go.

It didn't work. This is a (epic boring)  story of how I debugged and fixed the issues.

Step 1: update

Update your neovim to the latest release to avoid fighting issues that have already been solved. At the time of writing, I used:
  • nvim 0.1.2 from Homebrew
  • OS X 10.10.5
  • XCode 7.0
  • lldb-340.4.70

Step 2: Diagnose

PyThreadState_get error

If you're on OS X, chances are you have more than one Python version installed and that's where the trouble comes from. If you get this error message

>>> import lldb
Fatal Python error: PyThreadState_Get: no current thread

it's most likely because you're trying to import a module that has been linked with a different version of Python. The lldb module comes with the XCode developer tools and was linked with the default system version of Python which lives in (remember this)

/System/Library/Frameworks/Python.framework/Versions/2.7/bin/python

so this is the python version you should use to run the lldb.nvim remote plugin. On my system, BTW, lldb module lives in

/Applications/Xcode.app/Contents/SharedFrameworks/LLDB.framework/Resources/Python

Step 3: Install neovim Python module

The neovim module has probably already been installed with neovim but perhaps not in the correct Python version. You can try to  import neovim in the system Python. If it fails, you'll need to install it using easy_install or pip:

sudo /System/Library/Frameworks/Python.framework/Versions/2.7/bin/python -m easy_install neovim

This will install the neovim package into the system Python distribution (needs sudo) using the easy_install tool.

Step 4: Configure neovim to use the system Python

In your neovim config file, add this line:

let g:python_host_prog = '/System/Library/Frameworks/Python.framework/Versions/2.7/bin/python'

This ensures that neovim will start the system Python (which has access to lldb and neovim modules) to host the plugin. After this, you should be all set!

Step 5: Using $PYTHONPATH?

If you do use $PYTHONPATH with your non-system Python, you'll have trouble as well. Before launching the system Python from nvim, you'll need to clean this variable otherwise the packages will interfere with the system Python's packages.

I do that using a small wrapper script ~/syspython2 which gets invoked from nvim as the g:python_host_prog

#!/bin/sh
# running the OS X system python. Required to import the lldb module.
export PYTHONPATH="/Applications/Xcode.app/Contents/SharedFrameworks/LLDB.framework/Resources/Python"

# enable these for debugging
#echo "--" >> ~/syspython2.log
#echo "$@" >> ~/syspython2.log
/System/Library/Frameworks/Python.framework/Versions/2.7/bin/python "$@"

Step 6: Diagnose

Still having trouble?

  • Don't forget to run :UpdateRemotePlugins
  • Enable logging in the ~/syspython2 script
  • Check using pstree | less if neovim is launching the correct Python binary
  • Double-check you can import neovim and lldb modules from the system Python
  • Make sure lldb.neovim is installed correctly - the file lldb.nvim/rplugin/python/lldb_nvim.py must exist
  • NeoVim also tries to load Python 3 plugins, you may need to do the same for Python 3
  • Try to debug /usr/local/Cellar/neovim/0.1.2/share/nvim/runtime/autoload/remote/host.vim  using debugging vim methods
  • More info about lldb Python module here on StackOverflow
Posted by at
Labels: , , ,

12 comments:

  1. The YokerAugust 24, 2016 at 6:56 AM

    Thanks a million trillion times! I was getting suicidal,
    Here's one hundred smilies:
    :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :) :)

    ReplyDelete
  2. UnknownNovember 17, 2016 at 4:52 AM

    lldb works smooth with the default macOS python (/usr/bin/python). Which is also used in LLDB Python Reference: http://lldb.llvm.org/python-reference.html
    But there is a need to install all the dependencies and even pip itself for this version. My workaround was:
    1) installing pip with /usr/bin/python using get-pip.py (no luck with easy_install):
    download https://bootstrap.pypa.io/get-pip.py
    run the downloaded: sudo /usr/bin/python get-pip.py
    2) installing the 'six' module (sudo /usr/bin/python -m pip install six)
    3) installing neovim, but 'pip install' for each dependency:
    sudo /usr/bin/python -m pip install greenlet
    sudo /usr/bin/python -m pip install msgpack-python
    sudo /usr/bin/python -m pip install futures
    sudo /usr/bin/python -m pip install trollius
    sudo /usr/bin/python -m pip install neovim
    4) adding 'export PYTHONPATH=/Developer/Library/PrivateFrameworks/LLDB.framework/Resources/Python' to my ~/.bash_profile
    5) setting nvim's python to /usr/bin/python
    let g:python_host_prog = '/usr/bin/python'
    But this is clean Python on my system, all the modules that I previously installed is in /System/Library/Frameworks/Python.framework/... so it's not so great.

    ReplyDelete
    Replies
    1. UnknownNovember 19, 2016 at 6:07 AM

      Sorry, PYTHONPATH should be set as:
      PYTHONPATH=/Applications/Xcode.app/Contents/SharedFrameworks/LLDB.framework/Resources/Python

      Delete
  3. UnknownDecember 6, 2016 at 4:31 AM

    Thanks for your great tutorial. I had problem with addressing python host like this:
    let g:python_host_prog = '~/syspython2'
    This should be:
    let g:python_host_prog = '/Users/username/syspython2'
    With this configuration it works :)

    ReplyDelete
  4. CrebsMay 4, 2017 at 9:57 AM

    Thank you. Excited to use lldb in neovim now!

    ReplyDelete
  5. marksonSeptember 16, 2019 at 12:55 AM

    This huge change supports and actuating the Data Scientist over the association to adapt Big Data Training in Chennai. ExcelR Data Science Courses

    ReplyDelete
  6. datasciencecourseFebruary 17, 2020 at 8:39 PM

    After reading your article I was amazed. I know that you explain it very well. And I hope that other readers will also experience how I feel after reading your article.

    data analytics courses

    data science interview questions

    business analytics courses

    data science course in mumbai

    ReplyDelete
  7. Analytics pathMarch 17, 2020 at 3:52 AM

    Inspirational Blog The Content Mentioned is Knowledgeable and Very Effective .Thanks For Sharing It

    AI Training In Hyderabad

    ReplyDelete
  8. shanjamesMarch 21, 2020 at 12:05 AM

    The concept has been explained very well. Gained clear knowledge about the prominence of Data Science.
    Data Sicence Training in Hyderabad

    ReplyDelete
  9. Kevin HorganMarch 28, 2020 at 2:41 AM

    When you are away from your office do you need to get access to your work desktop from time to time. Many people run into this problem and are forced ycombinator startup templates

    ReplyDelete
  10. shanjamesMarch 29, 2020 at 11:56 PM

    Nice blog Post ! This post contains very informative and knowledgeable. Thanks for sharing the most valuable information.
    Hadoop Training in Hyderabad

    ReplyDelete

Subscribe to: Post Comments (Atom)