Getting Started Remote Debugging Python Apps in PyCharm

Python really needs no introduction, but Google’s search algorithms demand it. Python is essential for its versatility and low barrier of entry for new users. It’s become a vital part of nearly every corner of development, especially in big data and machine learning which may not have progressed to where they are in 2022 without Python’s inception. All the same, debugging Python needs to be efficient. There are a prolific number of development environments and frameworks for Python users, all with their own advantages or particular users in mind. One of those IDEs is PyCharm, part of JetBrain’s arsenal of tools.

As is standard these days, you need to be able to handle remote debugging in whatever environment you choose. More often than not you’re going to have some part of your overall architecture living in the cloud. Just like with its signature IDE IntelliJ, PyCharm offers templates and custom options for configuration, as well as running multiple debugs at a time.

There are two kinds of debugging to speak of: local or standard debugging (which is available in the community edition) and remote debugging (which is a premium-only feature). 

Demand for Cloud-Native Debugging

This tutorial will show off PyCharm’s built-in debugging capabilities. To make this clear, we’ll work with local debugging. Local debugs are obviously simpler and doesn’t deal with some of the unspoken challenges inherent to remote debugging. Those challenges make it difficult – if not impossible – to adequately duplicate the conditions of your remote deployment.

Because of the complex architecture of remote microservices deployments, mimicking the conditions of the production deployment are nearly impossible. The demands for accurate debugging here require tools like Rookout that can conduct live debugging while syncing with the source code accurately. Third-party tools can also re-secure information that might otherwise be accessed in a straight-up remote debug.

Python’s main debugging tool is pdb, including breakpoints and source code listing. It’s also extensible where pdb actually constitutes its own class. You can run it automatically whenever you hit an error, or manually call with a command as simple as this:

import pdb

python -m pdb mystuff.py

This tutorial will show off PyCharm’s built-in debugging capabilities. To make this clear, we’ll work with local debugging. Remote debugging deserves its own attention as the demands for accurate debugging include tools like Rookout that can conduct live debugging while properly duplicating all the conditions inherent to your production environment.

Let’s set up a simple app and run it through a debugging session in PyCharm to see how it works.

Setting Up Remote Python Debugging in PyCharm

Prerequisites

• Python project
• PyCharm community edition
• Python 3+

You’re going to want to have the most advanced version of Python possible. Since I only have Python 3.9, I’m going to check on going to update to whatever the latest version of 3.10 is. On a Mac, you’re probably using Homebrew, but make sure to specify brew install python@3.10, otherwise, it will upgrade you to the last version of Python 3. As of this writing, Homebrew offers up to 3.10.4 rather than 3.10.5, but these numbers are certain to change as time moves on. For those of you from the FUTURE, specify python@3.11, python@3.12, etc. to be as up-to-date as possible.

brew install python@3.10

<i><span style="font-weight: 400;">#Note - Those of you reading </span></i><i><span style="font-weight: 400;">from</span></i><i><span style="font-weight: 400;"> the </span></i><i><span style="font-weight: 400;">FUTURE</span></i><i><span style="font-weight: 400;"> year </span></i><i><span style="font-weight: 400;">2369</span></i><i><span style="font-weight: 400;">, you can either use Python </span></i><i><span style="font-weight: 400;">147.12.33.001</span></i> <i><span style="font-weight: 400;">or</span></i> <a href="https://www.rookout.com/blog/why-python-2-will-never-die/"><i><span style="font-weight: 400;">Python 2.7</span></i></a>

Download the PyCharm Community Edition from the JetBrains website. If you’re on a Mac, make sure to specify if you’ve got an Intel chip or an Apple Silicon chip (most will have Intel, but check the little apple in the upper lefthand corner of your screen). Download the .dmg file. The installation is easy from there. Once prompted, move PyCharm to your Applications folder and open it from there.

Step 1: Create a PyCharm Project

Create a project and save it. This is where you will create your own files in Step 2.

Now, create a new virtual environment and choose Virtualenv. Next, choose the venv location and its base interpreter (Python version).

At the bottom of the dialog box, deselect ☐ Create a main.py welcome script (since, you know, you’re going to create your own core .py files for the project in Step 2).

You can rename the main.py file if you’d like to match your project. In any case, that’s where your primary code will live for this simple app.

Step 2: Edit the code

This app will simply count the number of words in a given document. But to make it more interesting, we’ll add a frequency counter to tell us which words come up the most.

To keep this as relevant as possible, I took the text from Wikipedia’s article on debugging and copied it into a file debuggingwiki.txt.

We’ll set two functions to…

1. Count the total words and
2. Break that vocab down into a list with the number of occurrences for each word in the doc.

Define the functions’ output, then put in the (relatively) complicated stuff. To get the relevant files for this tutorial, look up its GitHub repo.

Functions:

def totalcount():
print('Total Words:', len(per_word))

def wordsbynum():
print('Word Frequency:', counts)

Code:

def totalcount():
print('Total Words:', len(per_word))

def wordsbynum():
print('Word Frequency:', counts)

file = open("debuggingwiki.txt", "r")
read_data = file.read()
per_word = read_data.split()

file = open("debuggingwiki.txt")
counts = dict()
for line in file:
words = line.split()
for word in words:
if word in counts:
counts[word] += 1
else:
counts[word] = 1

totalcount()
wordsbynum()

Run the code and you get:

Total Words: 2276
Word Frequency: {'Debugging': 7, 'In': 8, 'computer': 8, 'programming': 7, 'and': 52, 'software': 13, 'development,': 1, 'debugging': 25, 'is': 31, 'the': 141, 'process': 8, 'of': 77, 'finding': 1, 'resolving': 1, 'bugs': 5, …

#the list will continue for two more pages if I don't cut it off here, but don't worry, there's more!

It works! You’re a genius. You could beat a real python in a fight! But your code’s kind of discombobulated. Let’s try to improve the app by sorting this list in descending order.

Step 3: Debugging Pycharm Apps

Since you’re a pythonic genius now, let’s try your solution for sorting the word frequency list. Using the built-in function sorted(), you think you’ve got this:

You add in your favorite function: sort(). This will blow everyone away.

def inorder():
freqsort = sort.counts()
print(freqsort)

inorder()

Then run.

Total Words: 2276
Word Frequency: {'Debugging': 7, 'In': 8, 'computer': 8, 'programming': 7, 'and': 52, 'software': 13, 'development,': 1, 'debugging': 25, 'is': 31, 'the': 141, 'process': 8, 'of': 77, 'finding': 1, 'resolving': 1, 'bugs': 5, ...

<span style="text-color: red;">Traceback (most recent call last):
 File "/Users/gedalyahreback/PycharmProjects/demopycharm/practicescratch.py" line 35, in <module> inorder()
 File "/Users/gedalyahreback/PycharmProjects/demopycharm/practicescratch.py" line 32, in inorder
   freqsort = sort.counts()
NameError: name 'sort' is not defined</span>

Oh no, your code sucks now! What are you going to do?

No worries, that’s what debugging is for.

Either right click or hit the green bug icon in the upper-right-hand corner. If you want to set a breakpoint on any specific line, hit the space between the line number and the code itself.

You can and should set the debugger to stop when it hits an issue or a breakpoint. If it finds new issues, it will create and add a new breakpoint, signaling it with a lightning bolt (as seen here without the previously displayed red dots):

Restart the debug by hitting the resume icon: ⧐

In the debugger, it will mark the errors with more detail.

The sort() function is “not defined” because it’s acting on a dictionary . Unfortunately, sort() only works on lists.

Python Bonus Info: sort() vs. sorted()

Okay, fine, let’s try something else. We’ll go with sorted(), which accepts dictionaries.

Attempt 2:

def inorder():
  freqsort = sorted(counts, reverse=True)
  print(freqsort)

inorder()

But wait! Run for the output:

['-', 'zero', 'you', 'would', 'worse', 'world', 'working', 'words', 'wolf."', 'wolf', 'without', 'within', 'with.', 'with', 'will', 'whose', 'who', 'which,', 'which', 'whether', 'whereupon', 'where', 'when', 'what', 'were' .......

What in the wide, wide, world of sports? Where did the numbers go? In this case, a debug won’t help you. But don’t worry, the issue here is you need to order by the values in the dictionary, otherwise it will just return things according to the keys.

You can simplify this by importing an operator called itemgetter. Now, you will be able to quickly specify the items you want:

from operator import itemgetter

def inorder():
    for key, value in sorted(counts.items(), key=itemgetter(1), reverse=True):
        print(key,':', value)

Run and get this:

You’re a genius again!

With all the debugs and rewrites out of the way, let’s see our app code in all its glory:

from operator import itemgetter

def totalcount():
  print('Total Words:', len(per_word))

def wordsbynum():
  print('Word Frequency:', counts)

def inorder() -> object:
  for key, value in sorted(counts.items(), key=itemgetter(1), reverse=True):
      print(key,':', value)

file = open("debuggingwiki.txt", "r")
read_data = file.read()
per_word = read_data.split()

file = open("debuggingwiki.txt")
counts = dict()
for line in file:
  words = line.split()
  for word in words:
      if word in counts:
          counts[word] += 1
      else:
          counts[word] = 1

totalcount()
wordsbynum()
print('In order:')
inorder()

Standard Debugging vs. Remote Python Debugging in PyCharm

PyCharm really is fantastic. It provides a lot of the tools you need to push along, even for a Python novice like yours truly. PyCharm and other IDEs also offer remote debugging, but it leaves something to be desired. Besides the fact remote debugging is a premium feature on PyCharm, it’s not necessarily an effective way to spend your budget.

There are two ways to do a remote debug in PyCharm: 1) create a deployment config for a remote interpreter, or 2) Create a remote debug server config for local debugging.

There are quite a few drawbacks to remote debugging in general regardless of IDE, even though you’d expect it to be an efficient process by this point in the history of cloud computing and DevOps. 

The requirements for remote debugging are extensive. If these aren’t all in perfect alignment, a reliable debug won’t be possible:

1. No Room for Latency: Poor network connections on either end of the debug will surely slow down the process, but could make it unreliable altogether
2. No Source Code Sync: It’s nearly impossible to keep source code in sync during debugs, a problem that high latency would intensify. Microservices architecture is difficult to debug in general.
3. No Breakpoints: On PyCharm or IntelliJ, it’s recommended NOT to use breakpoints during production debugging because of the risk to the application
4. Admin-only access: This limits the list of people who can manage a debug, hampering team productivity
5. Information risk: Exposing code here can also increase the exposure of passwords and tokens. Rookout can’t actually read the code when it imports it, keeping secure whatever data the original source code holds.
6. Remote debugging with Kubernetes is inefficient: There’s no way to stress this enough. You need to change the Dockerfile and redeploy the Docker image every … single … time  you need to do remote debugging.

Conclusion

Remote debugging usually requires extra care with version control and mimicking the exact conditions of a cloud – or even multi-cloud – deployment. Your connection must be extremely consistent and strong, which could hamper some efforts at doing timely debugging if you sometimes work from home where the connection could be weaker than your office setup. With all this in mind, Rookout is designed to mitigate these issues with our Live Debugger platform. Take a look and try Live Debugger for free.